公开(公告)号：US10366538B2

公开(公告)日：2019-07-30

申请号：US14887280

申请日：2015-10-19

Applicant: Apple Inc.

Inventor： Peter Meier ,  Stefan Holzer

IPC: G06T7/73 ,  G06T15/04 ,  G06T15/20 ,  G06T19/00

Abstract: The invention relates to a method for representing a virtual object in a real environment, having the following steps: generating a two-dimensional image of a real environment by means of a recording device, ascertaining a position of the recording device relative to at least one component of the real environment, segmenting at least one area of the real environment unmarked in reality in the two-dimensional image for identifying at least one segment of the real environment in distinction to a remaining part of the real environment while supplying corresponding segmentation data, and merging the virtual object with the two-dimensional image of the real environment with consideration of the segmentation data such that at least one part of the segment of the real environment is removed from the image of the real environment. The invention permits any collisions of virtual objects with real objects that occur upon merging with a real environment to be represented in a way largely close to reality.

公开(公告)号：US20190180512A1

公开(公告)日：2019-06-13

申请号：US16268623

申请日：2019-02-06

Applicant: Apple Inc.

Inventor： Anton Fedosov ,  Stefan Misslinger ,  Peter Meier

IPC: G06T19/00 ,  H04L29/06 ,  G06T15/20 ,  G01C21/36 ,  G06F3/01

Abstract: There is disclosed a method and mobile device for displaying points of interest in a view of a real environment displayed on a screen of the mobile device with a functionality for interaction with a user, which comprises the steps of: capturing an image of the real environment or a part of the real environment using a camera, determining at least one point of interest related to the real environment, determining an image position of the at least one point of interest in the image, displaying at least part of the image on at least part of the screen, overlaying a computer-generated indicator with the at least part of the image on the screen at a screen position according to the image position of the at least one point of interest, displaying a computer-generated virtual object related to the at least one point of interest on the screen at a screen position determined according to the screen position of the computer-generated indicator and which is adjacent to a bottom edge of the screen, displaying a visually perceivable relation indication indicative of a relation between the computer-generated virtual object and the computer-generated indicator. The mobile device may perform an action related to the at least one point of interest if at least part of the computer-generated virtual object displayed on the screen is touched.

公开(公告)号：US20190172220A1

公开(公告)日：2019-06-06

申请号：US16258110

申请日：2019-01-25

Applicant: Apple Inc.

Inventor： Peter Meier

IPC: G06T7/73 ,  G06K9/00 ,  G06K9/32 ,  G06T7/246 ,  G06T7/33 ,  G06K9/46 ,  G06T7/30 ,  G06K9/62 ,  G06T7/70

Abstract: A method for determining the pose of a camera relative to a real environment includes the following steps: taking at least one image of a real environment by means of a camera, the image containing at least part of a real object, performing a tracking method that evaluates information with respect to correspondences between features associated with the real object and corresponding features of the real object as it is contained in the image of the real environment, so as to obtain conclusions about the pose of the camera, determining at least one parameter of an environmental situation, and performing the tracking method in accordance with the at least one parameter. Analogously, the method can also be utilized in a method for recognizing an object of a real environment in an image taken by a camera.

公开(公告)号：US10229511B2

公开(公告)日：2019-03-12

申请号：US14977288

申请日：2015-12-21

Applicant: Apple Inc.

Inventor： Peter Meier

IPC: G06K9/00 ,  G06T7/73 ,  G06K9/32 ,  G06K9/46 ,  G06K9/62 ,  G06T7/30 ,  G06T7/33 ,  G06T7/70 ,  G06T7/246

Abstract: A method for determining the pose of a camera relative to a real environment includes the following steps: taking at least one image of a real environment by means of a camera, the image containing at least part of a real object, performing a tracking method that evaluates information with respect to correspondences between features associated with the real object and corresponding features of the real object as it is contained in the image of the real environment, so as to obtain conclusions about the pose of the camera, determining at least one parameter of an environmental situation, and performing the tracking method in accordance with the at least one parameter. Analogously, the method can also be utilized in a method for recognizing an object of a real environment in an image taken by a camera.

公开(公告)号：US20180307708A1

公开(公告)日：2018-10-25

申请号：US15919892

申请日：2018-03-13

Applicant: Apple Inc.

Inventor： Sebastian Lieberknecht ,  Peter Meier

IPC: G06F17/30 ,  G06K9/00

CPC classification number: G06F16/532 ,  G06F3/011 ,  G06F16/583 ,  G06F16/5838 ,  G06F16/93 ,  G06K9/00442 ,  G06K9/00671

Abstract: A method for analyzing an image of a real object, generated by at least one camera includes the following steps: generating at least a first image by the camera capturing at least one real object, defining a first search domain comprising multiple data sets of the real object, each of the data sets being indicative of a respective portion of the real object, and analyzing at least one characteristic property of the first image with respect to the first search domain, in order to determine whether the at least one characteristic property corresponds to information of at least a particular one of the data sets of the first search domain. If it is determined that the at least one characteristic property corresponds to information of at least a particular one of the data sets, a second search domain comprising only the particular one of the data sets is defined and the second search domain is used for analyzing the first image and/or at least a second image.

公开(公告)号：US10109107B2

公开(公告)日：2018-10-23

申请号：US14668381

申请日：2015-03-25

Applicant: Apple Inc.

Inventor： Sebastian Knorr ,  Peter Meier

IPC: G09G5/00 ,  G06T19/00 ,  G06T15/50

Abstract: A method and system of representing a virtual object in a view of a real environment is provided which includes providing image information of a first image of at least part of a human face captured by a camera, providing at least one human face specific characteristic, determining an image area of the face in the first image as a face region, determining at least one first light falling on the face according to the face region and the at least one human face specific characteristic, and blending in the virtual object on a display device in the view of the real environment according to at least one first light. The method further includes providing a plurality of face sample positions, providing for each of the plurality of face sample positions a radiance transfer function, determining image positions of the face sample positions in the first image, and determining a first light according to intensity information associated with the image positions of the face sample positions in the first image and the radiance transfer functions associated with the face sample positions in the first image.

公开(公告)号：US10074215B2

公开(公告)日：2018-09-11

申请号：US14616213

申请日：2015-02-06

Applicant: Apple Inc.

Inventor： Peter Meier ,  Michael Kuhn ,  Frank Angermann

IPC: G06T19/00

CPC classification number: G06T19/006 ,  G06T19/00 ,  G06T2219/024

Abstract: A method for representing virtual information in a view of a real environment comprises providing a virtual object having a global position and orientation with respect to a geographic global coordinate system, with first pose data on the global position and orientation of the virtual object, in a database of a server, taking an image of a real environment by a mobile device and providing second pose data as to at which position and with which orientation with respect to the geographic global coordinate system the image was taken. The method further includes displaying the image on a display of the mobile device, accessing the virtual object in the database and positioning the virtual object in the image on the basis of the first and second pose data, manipulating the virtual object or adding a further virtual object, and providing the manipulated virtual object with modified first pose data or the further virtual object with third pose data in the database.

公开(公告)号：US20180232942A1

公开(公告)日：2018-08-16

申请号：US15925432

申请日：2018-03-19

Applicant: Apple Inc.

Inventor： Lejing Wang ,  Peter Meier ,  Stefan Misslinger

IPC: G06T15/20 ,  G06T15/06 ,  G06T19/00

CPC classification number: G06T15/20 ,  G06T15/06 ,  G06T19/006

Abstract: A method for representing virtual information in a view of a real environment is provided that includes: providing a system setup including at least one display device, wherein the system setup is adapted for blending in virtual information on the display device in at least part of the view, determining a position and orientation of a viewing point relative to at least one component of the real environment, providing a geometry model of the real environment, providing at least one item of virtual information and a position of the at least one item of virtual information, determining whether the position of the item of virtual information is inside a 2D or 3D geometrical shape, determining a criterion which is indicative of whether the built-in real object is at least partially visible or non-visible in the view of the real environment, and blending in the at least one item of virtual information on the display device in at least part of the view of the real environment.

公开(公告)号：US20180089899A1

公开(公告)日：2018-03-29

申请号：US15713274

申请日：2017-09-22

Applicant: Apple Inc.

Inventor： Patrick S. Piemonte ,  Daniel De Rocha Rosario ,  Jason D. Gosnell ,  Peter Meier

IPC: G06T19/00 ,  G06T15/20 ,  G01C21/36 ,  G02B27/01 ,  G01C21/26

CPC classification number: G06T19/006 ,  G01C21/26 ,  G01C21/365 ,  G02B27/017 ,  G02B2027/0141 ,  G05D2201/0213 ,  G06T15/205 ,  G06T15/506 ,  G06T17/00 ,  G06T19/20 ,  G06T2210/56

Abstract: An AR system that leverages a pre-generated 3D model of the world to improve rendering of 3D graphics content for AR views of a scene, for example an AR view of the world in front of a moving vehicle. By leveraging the pre-generated 3D model, the AR system may use a variety of techniques to enhance the rendering capabilities of the system. The AR system may obtain pre-generated 3D data (e.g., 3D tiles) from a remote source (e.g., cloud-based storage), and may use this pre-generated 3D data (e.g., a combination of 3D mesh, textures, and other geometry information) to augment local data (e.g., a point cloud of data collected by vehicle sensors) to determine much more information about a scene, including information about occluded or distant regions of the scene, than is available from the local data.

公开(公告)号：US11900569B2

公开(公告)日：2024-02-13

申请号：US17957416

申请日：2022-09-30

Applicant: APPLE INC.

Inventor： Peter Meier ,  Daniel Kurz ,  Brian Chris Clark ,  Mohamed Selim Ben Himane

IPC: G06T5/00 ,  G06T7/73 ,  G06V10/60 ,  G06V40/16 ,  G06V20/10

CPC classification number: G06T5/005 ,  G06T7/75 ,  G06V10/60 ,  G06V20/10 ,  G06V40/171

Abstract: Various implementations disclosed herein include devices, systems, and methods that detect surfaces and reflections in such surfaces. Some implementations involve providing a CGR environment that includes virtual content that replaces the appearance of a user or the user's device in a mirror or other surface providing a reflection. For example, a CGR environment may be modified to include a reflection of the user that does not include the device that the user is holding or wearing. In another example, the CGR environment is modified so that virtual content, such as a newer version of the electronic device or a virtual wand, replaces the electronic device in the reflection. In another example, the CGR environment is modified so that virtual content, such as a user avatar, replaces the user in the reflection.