公开(公告)号：US11880951B2

公开(公告)日：2024-01-23

申请号：US17818255

申请日：2022-08-08

Applicant: Apple Inc.

Inventor： Peter Meier ,  Michael Kuhn ,  Frank Angermann

IPC: G06T19/00 ,  G06T7/73 ,  G01S19/01

CPC classification number: G06T19/006 ,  G06T7/74 ,  G06T19/00 ,  G01S19/01 ,  G06T2219/024

Abstract: A method for representing virtual information in a view of a real environment comprises providing a virtual object having a global position and orientation with respect to a geographic global coordinate system, with first pose data on the global position and orientation of the virtual object, in a database of a server, taking an image of a real environment by a mobile device and providing second pose data as to at which position and with which orientation with respect to the geographic global coordinate system the image was taken. The method further includes displaying the image on a display of the mobile device, accessing the virtual object in the database and positioning the virtual object in the image on the basis of the first and second pose data, manipulating the virtual object or adding a further virtual object, and providing the manipulated virtual object with modified first pose data or the further virtual object with third pose data in the database.

公开(公告)号：US11727606B2

公开(公告)日：2023-08-15

申请号：US17707774

申请日：2022-03-29

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Daniel Ulbricht ,  Jean-Daniel E. Nahmias ,  Omar Elafifi ,  Peter Meier

IPC: G06T11/00 ,  G06F3/04883

CPC classification number: G06T11/00 ,  G06F3/04883 ,  G06T2200/24

Abstract: In one implementation, a method includes: obtaining a user input to view SR content associated with video content; if the video content includes a first scene when the user input was detected: obtaining first SR content for a first time period of the video content associated with the first scene; obtaining a task associated with the first scene; and causing presentation of the first SR content and a first indication of the task associated with the first scene; and if the video content includes a second scene when the user input was detected: obtaining second SR content for a second time period of the video content associated with the second scene; obtaining a task associated with the second scene; and causing presentation of the second SR content and a second indication of the task associated with the second scene.

公开(公告)号：US11652965B2

公开(公告)日：2023-05-16

申请号：US17028360

申请日：2020-09-22

Applicant: Apple Inc.

Inventor： Peter Meier ,  Mohamed Selim Ben Himane ,  Daniel Kurz

IPC: H04N9/31 ,  H04N5/225 ,  H04N13/128 ,  G01B11/24 ,  G01B11/25 ,  H04N13/15 ,  H04N5/33 ,  H04N13/257 ,  H04N13/00

CPC classification number: H04N9/3185 ,  G01B11/24 ,  G01B11/25 ,  H04N5/2256 ,  H04N5/332 ,  H04N9/3191 ,  H04N9/3194 ,  H04N13/128 ,  H04N13/15 ,  H04N13/257 ,  H04N2013/0081

Abstract: A method of projecting digital information on a real object in a real environment includes the steps of projecting digital information on a real object or part of a real object with a visible light projector, capturing at least one image of the real object with the projected digital information using a camera, providing a depth sensor registered with the camera, the depth sensor capturing depth data of the real object or part of the real object, and calculating a spatial transformation between the visible light projector and the real object based on the at least one image and the depth data. The invention is also concerned with a corresponding system.

公开(公告)号：US11386653B2

公开(公告)日：2022-07-12

申请号：US16961835

申请日：2019-01-18

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Daniel Ulbricht ,  Jean-Daniel E. Nahmias ,  Omar Elafifi ,  Peter Meier

IPC: G06T7/73 ,  G06V20/20 ,  G06T19/00

Abstract: In one implementation, a method includes: identifying a first plot-within a scene associated with a portion of video content; synthesizing a scene description for the scene that corresponds to a trajectory of the first plot-effectuator within a setting associated with the scene and actions performed by the first plot-effectuator; and generating a corresponding synthesized reality (SR) reconstruction of the scene by driving a first digital asset associated with the first plot-effectuator according to the scene description for the scene.

公开(公告)号：US11336867B2

公开(公告)日：2022-05-17

申请号：US16749262

申请日：2020-01-22

Applicant: Apple Inc.

Inventor： Peter Meier ,  Sebastian Lieberknecht

IPC: G06K9/00 ,  G09G5/00 ,  H04N7/18 ,  G06T7/579 ,  G06T7/246 ,  G06T17/10 ,  G06T7/70

Abstract: A method of tracking a mobile device comprising at least one camera in a real environment comprises the steps of receiving image information associated with at least one image captured by the at least one camera, generating a first geometrical model of at least part of the real environment based on environmental data or mobile system state data acquired in an acquisition process by at least one sensor of a mobile system, which is different from the mobile device, and performing a tracking process based on the image information associated with the at least one image and at least partially according to the first geometrical model, wherein the tracking process determines at least one parameter of a pose of the mobile device relative to the real environment. The invention is also related to a method of generating a geometrical model of at least part of a real environment using image information from at least one camera of a mobile device.

公开(公告)号：US20220004767A1

公开(公告)日：2022-01-06

申请号：US17482133

申请日：2021-09-22

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Mohamed Selim Ben Himane ,  Peter Meier

IPC: G06K9/00 ,  G06T7/70 ,  G06T11/00

Abstract: In one implementation, a method includes: obtaining image data from an image sensor; recognizing a portion of an object within the image data; obtaining synthesized reality (SR) content associated with the portion of the object; and displaying the SR content in association with the portion of the object. In some implementations, the SR content is dependent on the orientation of an electronic device or the user relative to the object. In some implementations, the SR content is generated based on sensor data associated with the object.

公开(公告)号：US11182961B2

公开(公告)日：2021-11-23

申请号：US16167156

申请日：2018-10-22

Applicant: Apple Inc.

Inventor： Sebastian Knorr ,  Peter Meier

IPC: G06T19/00 ,  G06T15/50 ,  H04N13/239 ,  G06T7/11

Abstract: A method and system of representing a virtual object in a view of a real environment is provided which includes providing image information of a first image of at least part of a human face captured by a camera, providing at least one human face specific characteristic, determining an image area of the face in the first image as a face region, determining at least one first light falling on the face according to the face region and the at least one human face specific characteristic, and blending in the virtual object on a display device in the view of the real environment according to at least one first light. The method further includes providing a plurality of face sample positions, providing for each of the plurality of face sample positions a radiance transfer function, determining image positions of the face sample positions in the first image, and determining a first light according to intensity information associated with the image positions of the face sample positions in the first image and the radiance transfer functions associated with the face sample positions in the first image.

公开(公告)号：US11087136B2

公开(公告)日：2021-08-10

申请号：US16370231

申请日：2019-03-29

Applicant: Apple Inc.

Inventor： Peter Meier ,  Michael J. Rockwell

IPC: G06K9/00

Abstract: An exemplary process for identifying a type of a physical environment amongst a plurality of types of physical environments is provided. The process includes obtaining, using the one or more cameras, image data corresponding to a physical environment. The process further includes identifying at least one portion of an entity in the physical environment based on the image data; determining, based on the identified at least one portion of the entity, whether the entity is an entity of a first type; determining a type of the physical environment if the entity is an entity of the first type; and presenting one or more virtual objects and a representation of the entity.

公开(公告)号：US20210166490A1

公开(公告)日：2021-06-03

申请号：US17175456

申请日：2021-02-12

Applicant: Apple Inc.

Inventor： Patrick S. Piemonte ,  Daniel De Rocha Rosario ,  Jason D. Gosnell ,  Peter Meier

IPC: G06T19/00 ,  G06T17/00 ,  G06T19/20 ,  G01C21/26 ,  G01C21/36 ,  G02B27/01 ,  G06T15/20

Abstract: An AR system that leverages a pre-generated 3D model of the world to improve rendering of 3D graphics content for AR views of a scene, for example an AR view of the world in front of a moving vehicle. By leveraging the pre-generated 3D model, the AR system may use a variety of techniques to enhance the rendering capabilities of the system. The AR system may obtain pre-generated 3D data (e.g., 3D tiles) from a remote source (e.g., cloud-based storage), and may use this pre-generated 3D data (e.g., a combination of 3D mesh, textures, and other geometry information) to augment local data (e.g., a point cloud of data collected by vehicle sensors) to determine much more information about a scene, including information about occluded or distant regions of the scene, than is available from the local data.

公开(公告)号：US20210074031A1

公开(公告)日：2021-03-11

申请号：US16963308

申请日：2019-01-18

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Daniel Ulbricht ,  Jean-Daniel E. Nahmias ,  Omar Elafifi ,  Peter Meier

IPC: G06T11/00

Abstract: In one implementation, a method includes: while causing presentation of video content having a current plot setting, receiving a user input indicating a request to explore the current plot setting; obtaining synthesized reality (SR) content associated with the current plot setting in response to receiving the user input; causing presentation of the SR content associated with the current plot setting; receiving one or more user interactions with the SR content; and adjusting the presentation of the SR content in response to receiving the one or more user interactions with the SR content.