公开(公告)号：US11423604B2

公开(公告)日：2022-08-23

申请号：US17397437

申请日：2021-08-09

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Maxime Meilland ,  Patrick W. O'Keefe

IPC: G06T19/00 ,  G06T15/08 ,  G02B27/01 ,  G06F3/01 ,  G06T17/20

Abstract: In one implementation, a method includes: obtaining locality data characterizing objects and relative spatial information of a volumetric region around a user; synthesizing a mesh map of the volumetric region based on the locality data; selecting synthesized reality (SR) content based on the mesh map, wherein the SR content satisfies a dimensional variance threshold relative to one or more portions of the mesh map; compositing at least a portion of the SR content with the mesh map in order to generate composite SR content; and presenting the composite SR content to the user in order to occlude at least a portion of a visual presentation of the volumetric region. In some implementations, the SR content is adapted to fit the one or more portions of the mesh map. In some implementations, the SR content is updated as the user location changes or the user interacts with the SR content.

公开(公告)号：US20220256249A1

公开(公告)日：2022-08-11

申请号：US17729462

申请日：2022-04-26

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Daniel Ulbricht ,  Eshan Verma

IPC: H04N21/81 ,  G06V20/20 ,  G06T19/00 ,  H04N21/44 ,  H04N21/845

Abstract: In one implementation, consumption of media content (such as video, audio, or text) is supplemented with an immersive synthesized reality (SR) map based on the media content. In various implementations described herein, the SR map includes a plurality of SR environment representations which, when selected by a user, cause display of a corresponding SR environment.

公开(公告)号：US11386653B2

公开(公告)日：2022-07-12

申请号：US16961835

申请日：2019-01-18

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Daniel Ulbricht ,  Jean-Daniel E. Nahmias ,  Omar Elafifi ,  Peter Meier

IPC: G06T7/73 ,  G06V20/20 ,  G06T19/00

Abstract: In one implementation, a method includes: identifying a first plot-within a scene associated with a portion of video content; synthesizing a scene description for the scene that corresponds to a trajectory of the first plot-effectuator within a setting associated with the scene and actions performed by the first plot-effectuator; and generating a corresponding synthesized reality (SR) reconstruction of the scene by driving a first digital asset associated with the first plot-effectuator according to the scene description for the scene.

公开(公告)号：US20220114795A1

公开(公告)日：2022-04-14

申请号：US17557739

申请日：2021-12-21

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Michael J. Rockwell ,  Amritpal Singh Saini ,  Olivier Soares

IPC: G06T19/00 ,  G06N3/04 ,  G06N3/08

Abstract: In one implementation, a method includes: instantiating a first objective-effectuator (OE) associated with first attributes and a second OE associated with second attributes into a synthesized reality (SR) setting, wherein the first OE is encapsulated within the second OE; providing a first objective to the first OE based on the first and second attributes; providing a second objective to the second OE based on the second attributes, wherein the first and second objectives are associated with a time period between a first and second temporal points; generating a first set of actions for the first OE based on the first objective and a second set of actions for the second OE based on the second objective; and rendering for display the SR setting for the time period including the first set of actions performed by the first OE and the second set of actions performed by the second OE.

公开(公告)号：US11301050B2

公开(公告)日：2022-04-12

申请号：US17059814

申请日：2019-05-29

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Christopher Eubank ,  Tomlinson Holman

IPC: G06F3/048 ,  G06F3/01 ,  G06F3/04815 ,  G06T19/00 ,  G06F3/0482

Abstract: In various implementations, methods of presenting a synthesized reality (SR) user interface are disclosed, including moving a data item in SR user interface, regrouping data items in an SR user interface, selecting groups of data items in an SR user interface, and playing audio files represented by data items in an SR user interface.

公开(公告)号：US20220004767A1

公开(公告)日：2022-01-06

申请号：US17482133

申请日：2021-09-22

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Mohamed Selim Ben Himane ,  Peter Meier

IPC: G06K9/00 ,  G06T7/70 ,  G06T11/00

Abstract: In one implementation, a method includes: obtaining image data from an image sensor; recognizing a portion of an object within the image data; obtaining synthesized reality (SR) content associated with the portion of the object; and displaying the SR content in association with the portion of the object. In some implementations, the SR content is dependent on the orientation of an electronic device or the user relative to the object. In some implementations, the SR content is generated based on sensor data associated with the object.

公开(公告)号：US11120612B2

公开(公告)日：2021-09-14

申请号：US16960163

申请日：2019-01-18

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Maxime Meilland ,  Patrick W. O'Keefe

IPC: G06T19/00 ,  G06T15/08 ,  G02B27/01 ,  G06F3/01 ,  G06T17/20

Abstract: In one implementation, a method includes: obtaining locality data characterizing objects and relative spatial information of a volumetric region around a user; synthesizing a mesh map of the volumetric region based on the locality data; selecting synthesized reality (SR) content based on the mesh map, wherein the SR content satisfies a dimensional variance threshold relative to one or more portions of the mesh map; compositing at least a portion of the SR content with the mesh map in order to generate composite SR content; and presenting the composite SR content to the user in order to occlude at least a portion of a visual presentation of the volumetric region. In some implementations, the SR content is adapted to fit the one or more portions of the mesh map. In some implementations, the SR content is updated as the user location changes or the user interacts with the SR content.

公开(公告)号：US20210233323A1

公开(公告)日：2021-07-29

申请号：US17231917

申请日：2021-04-15

Applicant: Apple Inc.

Inventor： Gutemberg B. Guerra Filho ,  Ian M. Richter ,  Raffi A. Bedikian

IPC: G06T19/00 ,  G06T19/20 ,  G06K9/00

Abstract: In some implementations, a method includes: determining first usage patterns associated with a physical object within the physical environment; obtaining a first objective for an objective-effectuator (OE) instantiated in a computer-generated reality (CGR) environment, wherein the first objective is associated with a representation of the physical object; obtaining a first directive for the OE that limits actions for performance by the OE to achieve the first objective to the first usage patterns associated with the physical object; generating first actions, for performance by the OE, in order to achieve the first objective as limited by the first directive, wherein the first set of actions corresponds to a first subset of usage patterns from the first set of usage patterns associated with the physical object; and presenting the OE performing the first actions on the representation of the physical object overlaid on the physical environment.

公开(公告)号：US20210074031A1

公开(公告)日：2021-03-11

申请号：US16963308

申请日：2019-01-18

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Daniel Ulbricht ,  Jean-Daniel E. Nahmias ,  Omar Elafifi ,  Peter Meier

IPC: G06T11/00

Abstract: In one implementation, a method includes: while causing presentation of video content having a current plot setting, receiving a user input indicating a request to explore the current plot setting; obtaining synthesized reality (SR) content associated with the current plot setting in response to receiving the user input; causing presentation of the SR content associated with the current plot setting; receiving one or more user interactions with the SR content; and adjusting the presentation of the SR content in response to receiving the one or more user interactions with the SR content.

公开(公告)号：US20200342231A1

公开(公告)日：2020-10-29

申请号：US16959253

申请日：2019-01-18

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Mohamed Selim Ben Himane ,  Peter Meier

IPC: G06K9/00 ,  G06T7/70 ,  G06T11/00

Abstract: In one implementation, a method includes: obtaining image data from an image sensor; recognizing a portion of an object within the image data; obtaining synthesized reality (SR) content—such as mixed reality, augmented reality, augmented virtuality, or virtual reality content—associated with the portion of the object; and displaying the SR content in association with the portion of the object. In some implementations, the SR content is dependent on the orientation of an electronic device or the user relative to the object. In some implementations, the SR content is generated based on sensor data associated with the object.