公开(公告)号：US12148066B2

公开(公告)日：2024-11-19

申请号：US18215311

申请日：2023-06-28

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Daniel Ulbricht ,  Jean-Daniel E. Nahmias ,  Omar Elafifi ,  Peter Meier

IPC: G06T11/00 ,  G06F3/04883

Abstract: In one implementation, a method includes: obtaining a user input to view SR content associated with video content; if the video content includes a first scene when the user input was detected: obtaining first SR content for a first time period of the video content associated with the first scene; obtaining a task associated with the first scene; and causing presentation of the first SR content and a first indication of the task associated with the first scene; and if the video content includes a second scene when the user input was detected: obtaining second SR content for a second time period of the video content associated with the second scene; obtaining a task associated with the second scene; and causing presentation of the second SR content and a second indication of the task associated with the second scene.

公开(公告)号：US20230290083A1

公开(公告)日：2023-09-14

申请号：US18153746

申请日：2023-01-12

Applicant: Apple Inc.

Inventor： Peter Meier ,  Frank A. Angermann

IPC: G06T19/00 ,  G06T15/20 ,  G06F3/01

CPC classification number: G06T19/006 ,  G06T15/20 ,  G06T19/00 ,  G06F3/011 ,  G01C23/00

Abstract: The invention relates to a method for ergonomically representing virtual information in a real environment, including the following steps: providing at least one view of a real environment and of a system setup for blending in virtual information for superimposing with the real environment in at least part of the view, the system setup comprising at least one display device, ascertaining a position and orientation of at least one part of the system setup relative to at least one component of the real environment, subdividing at least part of the view of the real environment into a plurality of regions comprising a first region and a second region, with objects of the real environment within the first region being placed closer to the system setup than objects of the real environment within the second region, and blending in at least one item of virtual information on the display device in at least part of the view of the real environment, considering the position and orientation of said at least one part of the system setup, wherein the virtual information is shown differently in the first region than in the second region with respect to the type of blending in the view of the real environment.

公开(公告)号：US11699270B2

公开(公告)日：2023-07-11

申请号：US17740052

申请日：2022-05-09

Applicant: Apple Inc.

Inventor： Bo Morgan ,  Mark E. Drummond ,  Peter Meier ,  Cameron J. Dunn ,  John Christopher Russell ,  Siva Chandra Mouli Sivapurapu ,  Ian M. Richter

IPC: G06T19/00 ,  A63F13/55 ,  G06F40/14 ,  G06F3/01

CPC classification number: G06T19/006 ,  A63F13/55 ,  G06F3/011 ,  G06F40/14

Abstract: In various implementations, a device surveys a scene and presents, within the scene, a extended reality (XR) environment including one or more assets that evolve over time (e.g., change location or age). Modeling such an XR environment at various timescales can be computationally intensive, particularly when modeling the XR environment over larger timescales. Accordingly, in various implementations, different models are used to determine the environment state of the XR environment when presenting the XR environment at different timescales.

公开(公告)号：US11694407B2

公开(公告)日：2023-07-04

申请号：US17170054

申请日：2021-02-08

Applicant: Apple Inc.

Inventor： Peter Meier

IPC: G06T19/00 ,  G02B27/01 ,  G06F3/01 ,  G06F3/14

CPC classification number: G06T19/006 ,  G02B27/017 ,  G06F3/011 ,  G06F3/14 ,  G02B2027/014 ,  G02B2027/0138 ,  G02B2027/0187

Abstract: A method of displaying virtual information in a view of a real environment comprising the following steps: providing the system relative to at least one part of the real environment and providing accuracy information of the current pose, providing multiple pieces of virtual information, and assigning a respective one of the pieces of virtual information to one of different parameters indicative of different pose accuracy information, and displaying at least one of the pieces of virtual information in the view of the real environment according to the accuracy information of the current pose in relation to the assigned parameter.

公开(公告)号：US11568760B1

公开(公告)日：2023-01-31

申请号：US16146909

申请日：2018-09-28

Applicant: Apple Inc.

Inventor： Peter Meier

IPC: G06K9/62 ,  G09B19/00 ,  G01N33/02 ,  G06N20/00 ,  G06V20/68 ,  G10L25/48

Abstract: Detecting a chewing noise from a user during a chewing session, triggering operation of a camera, obtaining image data capturing a food product, identifying the food product based on image data, determining a measurement of the chewing session, determining a volume of the food product based on the measurement of the chewing session, and determining a calorie intake based on the food product, the volume of the food product, and the measurement of the chewing session.

公开(公告)号：US11190753B1

公开(公告)日：2021-11-30

申请号：US16015833

申请日：2018-06-22

Applicant: Apple Inc.

Inventor： Peter Meier

IPC: H04N13/275 ,  G06T7/292 ,  G06T19/00 ,  H04N13/332 ,  G06K9/00 ,  H04N13/00

Abstract: Various implementations disclosed are for detecting moving objects that are in a field of view of a head-mountable device (HMD). In various implementations, the HMD includes a display, an event camera, a non-transitory memory, and a processor coupled with the display, the event camera and the non-transitory memory. In some implementations, the method includes synthesizing a first optical flow characterizing one or more objects in a field of view of the event camera based on depth data associated with the one or more objects. In some implementations, the method includes determining a second optical flow characterizing the one or more objects in the field of view of the event camera based on event image data provided by the event camera. In some implementations, the method includes determining that a first object of the one or more objects is moving based on the first optical flow and the second optical flow.

公开(公告)号：US11182974B2

公开(公告)日：2021-11-23

申请号：US16983629

申请日：2020-08-03

Applicant: Apple Inc.

Inventor： Sebastian Knorr ,  Peter Meier

IPC: G06T15/50 ,  H04N13/239 ,  G06T19/00 ,  G06T7/11

Abstract: The invention relates to a method of representing a virtual object in a view of a real environment which comprises the steps of providing image information of a first image of at least part of a human face captured by a first camera, providing at least one human face specific characteristic, determining at least part of an image area of the face in the first image as a face region of the first image, determining at least one first light falling on the face according to the face region of the first image and the at least one human face specific characteristic, and blending in the virtual object on a display device in the view of the real environment according to the at least one first light. The invention also relates to a system for representing a virtual object in a view of a real environment.

公开(公告)号：US10665025B2

公开(公告)日：2020-05-26

申请号：US16055907

申请日：2018-08-06

Applicant: Apple Inc.

Inventor： Peter Meier ,  Stefan Holzer

IPC: G09G5/02 ,  G06K9/36 ,  G06T19/00 ,  G06T15/20 ,  G06T7/70 ,  G06T7/10 ,  G06T19/20

Abstract: The invention relates to a method for representing a virtual object in a real environment, having the following steps: generating a two-dimensional representation of a real environment by means of a recording device, ascertaining a position of the recording device relative to at least one component of the real environment, segmenting at least one area of the real environment in the two-dimensional image on the basis of non-manually generated 3D information for identifying at least one segment of the real environment in distinction to a remaining part of the real environment while supplying corresponding segmentation data, and merging the two-dimensional image of the real environment with the virtual object or, by means of an optical, semitransparent element directly with reality with consideration of the segmentation data. The invention permits any collisions of virtual objects with real objects that occur upon merging with a real environment to be represented in a way largely close to reality.

公开(公告)号：US10607350B2

公开(公告)日：2020-03-31

申请号：US15618604

申请日：2017-06-09

Applicant: Apple Inc.

Inventor： Daniel Kurz ,  Peter Meier

IPC: G06K9/42 ,  G06K9/46 ,  G06T7/514 ,  G06T7/593 ,  G06T7/73

Abstract: The invention provides methods of detecting and describing features from an intensity image. In one of several aspects, the method comprises the steps of providing an intensity image captured by a capturing device, providing a method for determining a depth of at least one element in the intensity image, in a feature detection process detecting at least one feature in the intensity image, wherein the feature detection is performed by processing image intensity information of the intensity image at a scale which depends on the depth of at least one element in the intensity image, and providing a feature descriptor of the at least one detected feature. For example, the feature descriptor contains at least one first parameter based on information provided by the intensity image and at least one second parameter which is indicative of the scale.

公开(公告)号：US20190057554A1

公开(公告)日：2019-02-21

申请号：US16167156

申请日：2018-10-22

Applicant: Apple Inc.

Inventor： Sebastian Knorr ,  Peter Meier

IPC: G06T19/00 ,  G06T7/11 ,  H04N13/239 ,  G06T15/50

Abstract: A method and system of representing a virtual object in a view of a real environment is provided which includes providing image information of a first image of at least part of a human face captured by a camera, providing at least one human face specific characteristic, determining an image area of the face in the first image as a face region, determining at least one first light falling on the face according to the face region and the at least one human face specific characteristic, and blending in the virtual object on a display device in the view of the real environment according to at least one first light. The method further includes providing a plurality of face sample positions, providing for each of the plurality of face sample positions a radiance transfer function, determining image positions of the face sample positions in the first image, and determining a first light according to intensity information associated with the image positions of the face sample positions in the first image and the radiance transfer functions associated with the face sample positions in the first image.