公开(公告)号：US20210035367A1

公开(公告)日：2021-02-04

申请号：US16924559

申请日：2020-07-09

Applicant: Apple Inc.

Inventor： Eshan Verma ,  Daniel Ulbricht ,  Angela Blechschmidt ,  Mohammad Haris Baig ,  Chen-Yu Lee ,  Tanmay Batra

IPC: G06T19/00

Abstract: Various implementations disclosed herein include devices, systems, and methods that enable faster and more efficient real-time physical object recognition, information retrieval, and updating of a CGR environment. In some implementations, the CGR environment is provided at a first device based on a classification of the physical object, image or video data including the physical object is transmitted by the first device to a second device, and the CGR environment is updated by the first device based on a response associated with the physical object received from the second device.

公开(公告)号：US20210019949A1

公开(公告)日：2021-01-21

申请号：US17031676

申请日：2020-09-24

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Amit Kumar K C ,  Angela Blechschmidt ,  Chen-Yu Lee ,  Eshan Verma ,  Mohammad Haris Baig ,  Tanmay Batra

IPC: G06T19/00 ,  G02B27/01 ,  G06F3/01 ,  G06T7/11 ,  G06T7/30 ,  G06T7/50

Abstract: In one implementation, a method of generating a depth map is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes generating, based on a first image and a second image, a first depth map of the second image. The method includes generating, based on the first depth map of the second image and pixel values of the second image, a second depth map of the second image.

公开(公告)号：US20210012112A1

公开(公告)日：2021-01-14

申请号：US17032213

申请日：2020-09-25

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Angela Blechschmidt ,  Mohammad Haris Baig ,  Tanmay Batra ,  Eshan Verma ,  Amit Kumar KC

IPC: G06K9/00 ,  G06K9/62 ,  G06K9/72 ,  G06T19/00 ,  G06T7/70

Abstract: In one implementation, a method of generating a plane hypothesis is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method include obtaining a point cloud based on the image of the scene and generating an object classification set based on the image of the scene, each element of the object classification set including a respective plurality of pixels classified as a respective object in the scene. The method includes generating a plane hypothesis based on the point cloud and the object classification set.

公开(公告)号：US20190392213A1

公开(公告)日：2019-12-26

申请号：US16360732

申请日：2019-03-21

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Angela Blechschmidt ,  Mohammad Haris Baig ,  Tanmay Batra ,  Eshan Verma ,  Amit Kumar KC

IPC: G06K9/00 ,  G06K9/62 ,  G06K9/72 ,  G06T7/70 ,  G06T19/00

Abstract: In one implementation, a method of generating a plane hypothesis is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method include obtaining a point cloud based on the image of the scene and generating an object classification set based on the image of the scene, each element of the object classification set including a respective plurality of pixels classified as a respective object in the scene. The method includes generating a plane hypothesis based on the point cloud and the object classification set.

公开(公告)号：US12112446B1

公开(公告)日：2024-10-08

申请号：US18237480

申请日：2023-08-24

Applicant: Apple Inc.

Inventor： Angela Blechschmidt ,  Daniel Ulbricht ,  Alexander S. Polichroniadis

IPC: G06T19/20 ,  G06T7/70 ,  G06T17/00 ,  G06V20/70

CPC classification number: G06T19/20 ,  G06T7/70 ,  G06T17/00 ,  G06V20/70 ,  G06T2207/10028 ,  G06T2210/61 ,  G06T2219/2004

Abstract: Various implementations disclosed herein include devices, systems, and methods that uses object relationships represented in the scene graph to adjust the position of objects. For example, an example process may include obtaining a three-dimensional (3D) representation of a physical environment that was generated based on sensor data obtained during a scanning process, detecting positions of a set of objects in the physical environment based on the 3D representation, generating a scene graph for the 3D representation of the physical environment based on the detected positions of the set of objects, wherein the scene graph represents the set of objects and relationships between the objects, and determining a refined 3D representation of the physical environment by refining the position of at least one object in the set of objects based on the scene graph and an alignment rule associated with a relationship in the scene graph.

公开(公告)号：US12045941B1

公开(公告)日：2024-07-23

申请号：US17222571

申请日：2021-04-05

Applicant: Apple Inc.

Inventor： Payal Jotwani ,  Angela Blechschmidt

IPC: G06T19/00 ,  G06T7/13 ,  G06V10/44

CPC classification number: G06T19/006 ,  G06T7/13 ,  G06V10/443 ,  G06T2207/10028

Abstract: In one implementation, a method of generating an object model is performed at a device including one or more processors and non-transitory memory. The method includes obtaining a point cloud including a plurality of points, wherein each of the plurality of points is associated with a plurality of sets of coordinates in a three-dimensional space at a respective plurality of time periods. The method includes generating an object model including a plurality of vertices corresponding to the plurality of points and a plurality of edges between respective pairs of the plurality of vertices. The method includes generating, based on the sets of coordinates of a particular point, a characterization vector of a particular vertex corresponding to the particular point, wherein the characterization vector includes a joint label and joint data with respect to one or more of the plurality of edges of the particular vertex.

公开(公告)号：US12008720B1

公开(公告)日：2024-06-11

申请号：US17746218

申请日：2022-05-17

Applicant: Apple Inc.

Inventor： Bo Morgan ,  Payal Jotwani ,  Angela Blechschmidt ,  Mark E. Drummond ,  Daniel Ulbricht

IPC: G06T13/20 ,  G06T19/00

CPC classification number: G06T19/003 ,  G06T2200/24 ,  G06T2210/61

Abstract: In one implementation, a method of achieving an objective using a scene graph includes identifying a set of transition couples of a plurality of transition couples between sets of spatial relationships of the scene graph, wherein each spatial relationship of a respective first set of spatial relationships indicated by each transition couple of the set of transition couples is included in an initial set of spatial relationships or an respective second set of spatial relationships indicated by a previous transition couple and wherein a particular spatial relationship is included in a respective second set of spatial relationships of a last transition couple of the set of transition couples.

公开(公告)号：US11978248B1

公开(公告)日：2024-05-07

申请号：US18123103

申请日：2023-03-17

Applicant: Apple Inc.

Inventor： Angela Blechschmidt ,  Alexander S. Polichroniadis ,  Daniel Ulbricht

IPC: G06V20/10 ,  G06F18/20 ,  G06F18/22 ,  G06V20/40

CPC classification number: G06V20/10 ,  G06F18/22 ,  G06F18/29 ,  G06V20/46

Abstract: Implementations disclosed herein provide systems and methods that match a current relationship model associated with a user's current environment to a prior relationship model for a prior environment to determine that the user is in the same environment. The current relationship model is compared with the prior relationship model based on matching characteristics of the current relationship model with characteristics of the prior relationship model.

公开(公告)号：US11972607B2

公开(公告)日：2024-04-30

申请号：US18111541

申请日：2023-02-18

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Angela Blechschmidt ,  Mohammad Haris Baig ,  Tanmay Batra ,  Eshan Verma ,  Amit Kumar Kc

IPC: G06V20/10 ,  G06F18/24 ,  G06T7/70 ,  G06T19/00 ,  G06V30/262

CPC classification number: G06V20/10 ,  G06F18/24 ,  G06T7/70 ,  G06T19/006 ,  G06V30/274 ,  G06T2200/24 ,  G06T2207/10028 ,  G06T2207/20084

Abstract: In one implementation, a method of generating a plane hypothesis is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method includes obtaining a plurality of points of a point cloud based on the image of the scene. The method includes obtaining an object classification set based on the image of the scene. Each element of the object classification set includes a plurality of pixels respectively associated with a corresponding object in the scene. The method includes detecting a plane within the scene by identifying a subset of the plurality of points of the point cloud that correspond to a particular element of the object classification set.

公开(公告)号：US11783552B2

公开(公告)日：2023-10-10

申请号：US17557805

申请日：2021-12-21

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Amit Kumar K C ,  Angela Blechschmidt ,  Chen-Yu Lee ,  Eshan Verma ,  Mohammad Haris Baig ,  Tanmay Batra

IPC: G06T19/00 ,  G06F3/01 ,  A63F13/825 ,  G02B27/01 ,  A63F13/212 ,  G06F3/03

CPC classification number: G06T19/006 ,  A63F13/212 ,  A63F13/825 ,  G02B27/017 ,  G06F3/011 ,  G06F3/0304 ,  G06T19/003

Abstract: In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.