公开(公告)号：US09990726B2

公开(公告)日：2018-06-05

申请号：US14430170

申请日：2012-09-27

Applicant: Apple Inc.

Inventor： Marion Langer ,  Peter Meier ,  Selim BenHimane

IPC: G06T7/70 ,  G06T7/80 ,  G06T7/00 ,  H04N7/18 ,  G06K9/62 ,  H04N5/232 ,  G06K9/52

CPC classification number: G06T7/33 ,  G01S5/0247 ,  G01S5/163 ,  G01S19/53 ,  G06K9/3208 ,  G06K9/52 ,  G06K9/6201 ,  G06T5/00 ,  G06T7/251 ,  G06T7/70 ,  G06T7/75 ,  G06T7/80 ,  G06T2207/20036 ,  G06T2207/20101 ,  G06T2207/30244 ,  H04N5/23229 ,  H04N5/23293 ,  H04N7/18 ,  H04N17/002

Abstract: A method of determining a position and orientation of a device is provided. The position and orientation of the device is determined based on multiple degrees of freedom (DoF) and the device is associated with a capturing device for capturing at least one image is provided. The method includes: capturing at least one image of a real object with the capturing device, and providing a coordinate system in relation to the object; providing an estimation of intrinsic parameters of the capturing device; providing pose data to compute first and second DoFs in the coordinate system, with each DoF having a confidence degree; determining an initial pose of the device; performing a pose estimation process, and calculating in the pose estimation process an estimation of the DoFs having a second confidence degree; and determining a position and orientation of the device.

公开(公告)号：US09934612B2

公开(公告)日：2018-04-03

申请号：US14633386

申请日：2015-02-27

Applicant: Apple Inc.

Inventor： Peter Meier ,  Selim Ben Himane ,  Stefan Misslinger ,  Ben Blachnitzky

IPC: H04N7/18 ,  G06T19/00 ,  H04N5/222 ,  G06T15/00 ,  G06T7/73

CPC classification number: G06T19/006 ,  G06T7/75 ,  G06T15/00 ,  G06T2207/20092 ,  G06T2207/30244 ,  G06T2210/04 ,  H04N5/2224

Abstract: Method for determining the pose of a camera with respect to an object of a real environment for use in authoring/augmented reality application that includes generating a first image by the camera capturing a real object of a real environment, generating first orientation data from at least one orientation sensor associated with the camera or from an algorithm which analyzes the first image for finding and determining features which are indicative of an orientation of the camera, allocating a distance of the camera to the real object, generating distance data indicative of the allocated distance, determining the pose of the camera with respect to a coordinate system related to the real object of the real environment using the distance data and the first orientation data. May be performed with reduced processing requirements and/or higher processing speed, in mobile device such as mobile phones having display, camera and orientation sensor.

公开(公告)号：US09916330B2

公开(公告)日：2018-03-13

申请号：US15044009

申请日：2016-02-15

Applicant: Apple Inc.

Inventor： Sebastian Lieberknecht ,  Peter Meier

IPC: G06K9/00 ,  G06K9/46 ,  G06F17/30 ,  G06F3/01

CPC classification number: G06F17/30277 ,  G06F3/011 ,  G06F17/30011 ,  G06F17/30247 ,  G06F17/30256 ,  G06K9/00442 ,  G06K9/00671

Abstract: A method for analyzing an image of a real object, generated by at least one camera includes the following steps: generating at least a first image by the camera capturing at least one real object, defining a first search domain comprising multiple data sets of the real object, each of the data sets being indicative of a respective portion of the real object, and analyzing at least one characteristic property of the first image with respect to the first search domain, in order to determine whether the at least one characteristic property corresponds to information of at least a particular one of the data sets of the first search domain. If it is determined that the at least one characteristic property corresponds to information of at least a particular one of the data sets, a second search domain comprising only the particular one of the data sets is defined and the second search domain is used for analyzing the first image and/or at least a second image.

公开(公告)号：US20170109929A1

公开(公告)日：2017-04-20

申请号：US14887280

申请日：2015-10-19

Applicant: Apple Inc.

Inventor： Peter Meier ,  Stefan Holzer

IPC: G06T19/00 ,  G06T7/73 ,  G06T15/04

CPC classification number: G06T19/006 ,  G06T7/74 ,  G06T15/04 ,  G06T15/20 ,  G06T2207/30244 ,  G06T2210/04 ,  G06T2215/12 ,  G06T2215/16

Abstract: The invention relates to a method for representing a virtual object in a real environment, having the following steps: generating a two-dimensional image of a real environment by means of a recording device, ascertaining a position of the recording device relative to at least one component of the real environment, segmenting at least one area of the real environment unmarked in reality in the two-dimensional image for identifying at least one segment of the real environment in distinction to a remaining part of the real environment while supplying corresponding segmentation data, and merging the virtual object with the two-dimensional image of the real environment with consideration of the segmentation data such that at least one part of the segment of the real environment is removed from the image of the real environment. The invention permits any collisions of virtual objects with real objects that occur upon merging with a real environment to be represented in a way largely close to reality.

公开(公告)号：US11968479B2

公开(公告)日：2024-04-23

申请号：US17663533

申请日：2022-05-16

Applicant: Apple Inc.

Inventor： Peter Meier ,  Sebastian Lieberknecht

IPC: G06K9/00 ,  G06T7/246 ,  G06T7/579 ,  G06T7/70 ,  G06T17/10 ,  G09G5/00 ,  H04N7/18

CPC classification number: H04N7/185 ,  G06T7/251 ,  G06T7/579 ,  G06T7/70 ,  G06T17/10 ,  G06T2207/10016 ,  G06T2207/30244 ,  G06T2207/30264

Abstract: A method of tracking a mobile device comprising at least one camera in a real environment comprises the steps of receiving image information associated with at least one image captured by the at least one camera, generating a first geometrical model of at least part of the real environment based on environmental data or mobile system state data acquired in an acquisition process by at least one sensor of a mobile system, which is different from the mobile device, and performing a tracking process based on the image information associated with the at least one image and at least partially according to the first geometrical model, wherein the tracking process determines at least one parameter of a pose of the mobile device relative to the real environment. The invention is also related to a method of generating a geometrical model of at least part of a real environment using image information from at least one camera of a mobile device.

公开(公告)号：US11838492B2

公开(公告)日：2023-12-05

申请号：US17509349

申请日：2021-10-25

Applicant: Apple Inc.

Inventor： Peter Meier

IPC: H04N13/275 ,  G06T7/292 ,  G06T19/00 ,  H04N13/332 ,  G06V20/20 ,  H04N13/00

CPC classification number: H04N13/275 ,  G06T7/292 ,  G06T19/006 ,  G06V20/20 ,  H04N13/332 ,  H04N2013/0081

Abstract: Various implementations disclosed are for detecting moving objects that are in a field of view of a head-mountable device (HMD). In various implementations, the HMD includes a display, an event camera, a non-transitory memory, and a processor coupled with the display, the event camera and the non-transitory memory. In some implementations, the method includes synthesizing a first optical flow characterizing one or more objects in a field of view of the event camera based on depth data associated with the one or more objects. In some implementations, the method includes determining a second optical flow characterizing the one or more objects in the field of view of the event camera based on event image data provided by the event camera. In some implementations, the method includes determining that a first object of the one or more objects is moving based on the first optical flow and the second optical flow.

公开(公告)号：US11562540B2

公开(公告)日：2023-01-24

申请号：US16843462

申请日：2020-04-08

Applicant: Apple Inc.

Inventor： Peter Meier ,  Frank A. Angermann

IPC: G06T19/00 ,  G06T15/20 ,  G06F3/01 ,  G01C21/00 ,  G01C23/00

Abstract: The invention relates to a method for ergonomically representing virtual information in a real environment, including the following steps: providing at least one view of a real environment and of a system setup for blending in virtual information for superimposing with the real environment in at least part of the view, the system setup comprising at least one display device, ascertaining a position and orientation of at least one part of the system setup relative to at least one component of the real environment, subdividing at least part of the view of the real environment into a plurality of regions comprising a first region and a second region, with objects of the real environment within the first region being placed closer to the system setup than objects of the real environment within the second region, and blending in at least one item of virtual information on the display device in at least part of the view of the real environment, considering the position and orientation of said at least one part of the system setup, wherein the virtual information is shown differently in the first region than in the second region with respect to the type of blending in the view of the real environment.

公开(公告)号：US11462000B2

公开(公告)日：2022-10-04

申请号：US17001788

申请日：2020-08-25

Applicant: Apple Inc.

Inventor： Peter Meier ,  Daniel Kurz ,  Brian Chris Clark ,  Mohamed Selim Ben Himane

IPC: G06V10/60 ,  G06T7/73 ,  G06V10/145 ,  G06V40/16

Abstract: Various implementations disclosed herein include devices, systems, and methods that detect surfaces and reflections in such surfaces. Some implementations involve providing a CGR environment that includes virtual content that replaces the appearance of a user or the user's device in a mirror or other surface providing a reflection. For example, a CGR environment may be modified to include a reflection of the user that does not include the device that the user is holding or wearing. In another example, the CGR environment is modified so that virtual content, such as a newer version of the electronic device or a virtual wand, replaces the electronic device in the reflection. In another example, the CGR environment is modified so that virtual content, such as a user avatar, replaces the user in the reflection.

公开(公告)号：US20220279147A1

公开(公告)日：2022-09-01

申请号：US17663533

申请日：2022-05-16

Applicant: Apple Inc.

Inventor： Peter Meier ,  Sebastian Lieberknecht

IPC: H04N7/18 ,  G06T7/579 ,  G06T7/246 ,  G06T17/10 ,  G06T7/70

Abstract: A method of tracking a mobile device comprising at least one camera in a real environment comprises the steps of receiving image information associated with at least one image captured by the at least one camera, generating a first geometrical model of at least part of the real environment based on environmental data or mobile system state data acquired in an acquisition process by at least one sensor of a mobile system, which is different from the mobile device, and performing a tracking process based on the image information associated with the at least one image and at least partially according to the first geometrical model, wherein the tracking process determines at least one parameter of a pose of the mobile device relative to the real environment. The invention is also related to a method of generating a geometrical model of at least part of a real environment using image information from at least one camera of a mobile device

公开(公告)号：US11410391B2

公开(公告)日：2022-08-09

申请号：US16658851

申请日：2019-10-21

Applicant: Apple Inc.

Inventor： Peter Meier ,  Michael Kuhn ,  Frank Angermann

IPC: G06T19/00

Abstract: A method for representing virtual information in a view of a real environment comprises providing a virtual object having a global position and orientation with respect to a geographic global coordinate system, with first pose data on the global position and orientation of the virtual object, in a database of a server, taking an image of a real environment by a mobile device and providing second pose data as to at which position and with which orientation with respect to the geographic global coordinate system the image was taken. The method further includes displaying the image on a display of the mobile device, accessing the virtual object in the database and positioning the virtual object in the image on the basis of the first and second pose data, manipulating the virtual object or adding a further virtual object, and providing the manipulated virtual object with modified first pose data or the further virtual object with third pose data in the database.