公开(公告)号：US10922886B2

公开(公告)日：2021-02-16

申请号：US15713274

申请日：2017-09-22

Applicant: Apple Inc.

Inventor： Patrick S. Piemonte ,  Daniel De Rocha Rosario ,  Jason D Gosnell ,  Peter Meier

IPC: G06T19/00 ,  G06T17/00 ,  G06T19/20 ,  G01C21/26 ,  G01C21/36 ,  G02B27/01 ,  G06T15/20

Abstract: An AR system that leverages a pre-generated 3D model of the world to improve rendering of 3D graphics content for AR views of a scene, for example an AR view of the world in front of a moving vehicle. By leveraging the pre-generated 3D model, the AR system may use a variety of techniques to enhance the rendering capabilities of the system. The AR system may obtain pre-generated 3D data (e.g., 3D tiles) from a remote source (e.g., cloud-based storage), and may use this pre-generated 3D data (e.g., a combination of 3D mesh, textures, and other geometry information) to augment local data (e.g., a point cloud of data collected by vehicle sensors) to determine much more information about a scene, including information about occluded or distant regions of the scene, than is available from the local data.

公开(公告)号：US20200342231A1

公开(公告)日：2020-10-29

申请号：US16959253

申请日：2019-01-18

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Mohamed Selim Ben Himane ,  Peter Meier

IPC: G06K9/00 ,  G06T7/70 ,  G06T11/00

Abstract: In one implementation, a method includes: obtaining image data from an image sensor; recognizing a portion of an object within the image data; obtaining synthesized reality (SR) content—such as mixed reality, augmented reality, augmented virtuality, or virtual reality content—associated with the portion of the object; and displaying the SR content in association with the portion of the object. In some implementations, the SR content is dependent on the orientation of an electronic device or the user relative to the object. In some implementations, the SR content is generated based on sensor data associated with the object.

公开(公告)号：US10671662B2

公开(公告)日：2020-06-02

申请号：US15919892

申请日：2018-03-13

Applicant: Apple Inc.

Inventor： Sebastian Lieberknecht ,  Peter Meier

IPC: G06F16/532 ,  G06F16/93 ,  G06K9/00 ,  G06F3/01 ,  G06F16/583

Abstract: A method for analyzing an image of a real object, generated by at least one camera includes the following steps: generating at least a first image by the camera capturing at least one real object, defining a first search domain comprising multiple data sets of the real object, each of the data sets being indicative of a respective portion of the real object, and analyzing at least one characteristic property of the first image with respect to the first search domain, in order to determine whether the at least one characteristic property corresponds to information of at least a particular one of the data sets of the first search domain. If it is determined that the at least one characteristic property corresponds to information of at least a particular one of the data sets, a second search domain comprising only the particular one of the data sets is defined and the second search domain is used for analyzing the first image and/or at least a second image.

公开(公告)号：US20200162703A1

公开(公告)日：2020-05-21

申请号：US16749262

申请日：2020-01-22

Applicant: Apple Inc.

Inventor： Peter Meier ,  Sebastian Lieberknecht

IPC: H04N7/18 ,  G06T7/70 ,  G06T17/10 ,  G06T7/246 ,  G06T7/579

Abstract: A method of tracking a mobile device comprising at least one camera in a real environment comprises the steps of receiving image information associated with at least one image captured by the at least one camera, generating a first geometrical model of at least part of the real environment based on environmental data or mobile system state data acquired in an acquisition process by at least one sensor of a mobile system, which is different from the mobile device, and performing a tracking process based on the image information associated with the at least one image and at least partially according to the first geometrical model, wherein the tracking process determines at least one parameter of a pose of the mobile device relative to the real environment. The invention is also related to a method of generating a geometrical model of at least part of a real environment using image information from at least one camera of a mobile device

公开(公告)号：US20200160605A1

公开(公告)日：2020-05-21

申请号：US16658851

申请日：2019-10-21

Applicant: Apple Inc.

Inventor： Peter Meier ,  Michael Kuhn ,  Frank Angermann

IPC: G06T19/00

Abstract: A method for representing virtual information in a view of a real environment comprises providing a virtual object having a global position and orientation with respect to a geographic global coordinate system, with first pose data on the global position and orientation of the virtual object, in a database of a server, taking an image of a real environment by a mobile device and providing second pose data as to at which position and with which orientation with respect to the geographic global coordinate system the image was taken. The method further includes displaying the image on a display of the mobile device, accessing the virtual object in the database and positioning the virtual object in the image on the basis of the first and second pose data, manipulating the virtual object or adding a further virtual object, and providing the manipulated virtual object with modified first pose data or the further virtual object with third pose data in the database.

公开(公告)号：US10417824B2

公开(公告)日：2019-09-17

申请号：US15128418

申请日：2014-03-25

Applicant: Apple Inc.

Inventor： Sebastian Knorr ,  Peter Meier

IPC: G06T15/50 ,  H04N13/239 ,  G06T19/00 ,  G06T7/11

Abstract: The invention relates to a method of representing a virtual object in a view of a real environment which comprises the steps of providing image information of a first image of at least part of a human face captured by a first camera, providing at least one human face specific characteristic, determining at least part of an image area of the face in the first image as a face region of the first image, determining at least one first light falling on the face according to the face region of the first image and the at least one human face specific characteristic, and blending in the virtual object on a display device in the view of the real environment according to the at least one first light. The invention also relates to a system for representing a virtual object in a view of a real environment.

公开(公告)号：US10217288B2

公开(公告)日：2019-02-26

申请号：US15480142

申请日：2017-04-05

Applicant: Apple Inc.

Inventor： Anton Fedosov ,  Stefan Misslinger ,  Peter Meier

IPC: G06F3/01 ,  G01C21/36 ,  G06T19/00 ,  H04L29/06 ,  G06T15/20

Abstract: There is disclosed a method and mobile device for displaying points of interest in a view of a real environment displayed on a screen of the mobile device with a functionality for interaction with a user, which comprises the steps of: capturing an image of the real environment or a part of the real environment using a camera, determining at least one point of interest related to the real environment, determining an image position of the at least one point of interest in the image, displaying at least part of the image on at least part of the screen, overlaying a computer-generated indicator with the at least part of the image on the screen at a screen position according to the image position of the at least one point of interest, displaying a computer-generated virtual object related to the at least one point of interest on the screen at a screen position determined according to the screen position of the computer-generated indicator and which is adjacent to a bottom edge of the screen, displaying a visually perceivable relation indication indicative of a relation between the computer-generated virtual object and the computer-generated indicator. The mobile device may perform an action related to the at least one point of interest if at least part of the computer-generated virtual object displayed on the screen is touched.

公开(公告)号：US10043315B2

公开(公告)日：2018-08-07

申请号：US15205899

申请日：2016-07-08

Applicant: Apple Inc.

Inventor： Peter Meier ,  Stefan Holzer

IPC: G09G5/14 ,  G06T19/00 ,  G06T15/20 ,  G06T19/20 ,  G06T7/70 ,  G06T7/10

CPC classification number: G06T19/006 ,  G06T7/10 ,  G06T7/70 ,  G06T15/20 ,  G06T15/205 ,  G06T19/20 ,  G06T2207/20021 ,  G06T2207/30244 ,  G06T2210/04

Abstract: The invention relates to a method for representing a virtual object in a real environment, having the following steps: generating a two-dimensional representation of a real environment by means of a recording device, ascertaining a position of the recording device relative to at least one component of the real environment, segmenting at least one area of the real environment in the two-dimensional image on the basis of non-manually generated 3D information for identifying at least one segment of the real environment in distinction to a remaining part of the real environment while supplying corresponding segmentation data, and merging the two-dimensional image of the real environment with the virtual object or, by means of an optical, semitransparent element directly with reality with consideration of the segmentation data. The invention permits any collisions of virtual objects with real objects that occur upon merging with a real environment to be represented in a way largely close to reality.

公开(公告)号：US20180075620A1

公开(公告)日：2018-03-15

申请号：US15807263

申请日：2017-11-08

Applicant: Apple Inc.

Inventor： Lejing Wang ,  Peter Meier

IPC: G06T7/70

CPC classification number: G06T7/74 ,  G06T7/70 ,  G06T2207/10016 ,  G06T2207/30244

Abstract: The invention is related to a method and system for determining a pose of a first camera, comprising providing or receiving a spatial relationship (Rvc 1) between a visual content displayed on a display device and the first camera, receiving image information associated with an image (B1) of at least part of the displayed visual content captured by a second camera, and determining a pose of the first camera according to the image information associated with the image (B1) and the spatial relationship (Rvcl).

公开(公告)号：US09888235B2

公开(公告)日：2018-02-06

申请号：US14430169

申请日：2013-06-05

Applicant: Apple Inc.

Inventor： Peter Meier ,  Mohamed Selim Benhimane

IPC: G06K9/36 ,  H04N17/00 ,  G01S5/02 ,  G01S5/16 ,  H04N5/232 ,  G06K9/32 ,  G06K9/52 ,  G06T5/00 ,  H04N7/18 ,  G06T7/73 ,  G06T7/246 ,  G01S19/53

CPC classification number: G06T7/33 ,  G01S5/0247 ,  G01S5/163 ,  G01S19/53 ,  G06K9/3208 ,  G06K9/52 ,  G06K9/6201 ,  G06T5/00 ,  G06T7/251 ,  G06T7/70 ,  G06T7/75 ,  G06T7/80 ,  G06T2207/20036 ,  G06T2207/20101 ,  G06T2207/30244 ,  H04N5/23229 ,  H04N5/23293 ,  H04N7/18 ,  H04N17/002

Abstract: An image processing method includes the steps of providing at least one image of at least one object or part of the at least one object, and providing a coordinate system in relation to the image, providing at least one degree of freedom in the coordinate system or at least one sensor data in the coordinate system, and computing image data of the at least one image or at least one part of the at least one image constrained or aligned by the at least one degree of freedom or the at least one sensor data.