公开(公告)号：US20210035367A1

公开(公告)日：2021-02-04

申请号：US16924559

申请日：2020-07-09

Applicant: Apple Inc.

Inventor： Eshan Verma ,  Daniel Ulbricht ,  Angela Blechschmidt ,  Mohammad Haris Baig ,  Chen-Yu Lee ,  Tanmay Batra

IPC: G06T19/00

Abstract: Various implementations disclosed herein include devices, systems, and methods that enable faster and more efficient real-time physical object recognition, information retrieval, and updating of a CGR environment. In some implementations, the CGR environment is provided at a first device based on a classification of the physical object, image or video data including the physical object is transmitted by the first device to a second device, and the CGR environment is updated by the first device based on a response associated with the physical object received from the second device.

公开(公告)号：US20210019949A1

公开(公告)日：2021-01-21

申请号：US17031676

申请日：2020-09-24

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Amit Kumar K C ,  Angela Blechschmidt ,  Chen-Yu Lee ,  Eshan Verma ,  Mohammad Haris Baig ,  Tanmay Batra

IPC: G06T19/00 ,  G02B27/01 ,  G06F3/01 ,  G06T7/11 ,  G06T7/30 ,  G06T7/50

Abstract: In one implementation, a method of generating a depth map is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes generating, based on a first image and a second image, a first depth map of the second image. The method includes generating, based on the first depth map of the second image and pixel values of the second image, a second depth map of the second image.

公开(公告)号：US20210012112A1

公开(公告)日：2021-01-14

申请号：US17032213

申请日：2020-09-25

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Angela Blechschmidt ,  Mohammad Haris Baig ,  Tanmay Batra ,  Eshan Verma ,  Amit Kumar KC

IPC: G06K9/00 ,  G06K9/62 ,  G06K9/72 ,  G06T19/00 ,  G06T7/70

Abstract: In one implementation, a method of generating a plane hypothesis is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method include obtaining a point cloud based on the image of the scene and generating an object classification set based on the image of the scene, each element of the object classification set including a respective plurality of pixels classified as a respective object in the scene. The method includes generating a plane hypothesis based on the point cloud and the object classification set.

公开(公告)号：US20200074280A1

公开(公告)日：2020-03-05

申请号：US16513991

申请日：2019-07-17

Applicant: Apple Inc.

Inventor： Peter Meier ,  Tanmay Batra

IPC: G06N3/04 ,  G06N3/08 ,  G06K9/62 ,  G06T7/70 ,  G06T7/20

Abstract: In some implementations a neural network is trained to perform a main task using a clustering constraint, for example, using both a main task training loss and a clustering training loss. Training inputs are inputted into a main task neural network to produce output labels predicting locations of the parts of the objects in the training inputs. Data from pooled layers of the main task neural network is inputted into a clustering neural network. The main task neural network and the clustering neural network are trained based on a main task loss from the main task neural network and a clustering loss from the clustering neural network. The main task loss is determined by comparing differences between the output labels and the training labels. The clustering loss encourages the clustering network to learn to label the parts of the objects individually, e.g., to learn groups corresponding to the object parts.

公开(公告)号：US20200065653A1

公开(公告)日：2020-02-27

申请号：US16515397

申请日：2019-07-18

Applicant: Apple Inc.

Inventor： Peter Meier ,  Tanmay Batra

IPC: G06N3/04 ,  G06N3/08 ,  G06K9/62 ,  G06T7/70 ,  G06T7/20 ,  G06T7/10

Abstract: In some implementations at an electronic device, training a dual EDNN includes defining a data structure of attributes corresponding to defined parts of a task, processing a first instance of an input using a first EDNN to produce a first output while encoding a first set of the attributes in a first latent space, and processing a second instance of the input using a second EDNN to produce a second output while encoding attribute differences from attribute averages in a second latent space. The device then determines a second set of the attributes based on the attribute differences and the attribute averages. The device then adjusts parameters of the first and second EDNNs based on comparing the first instance of the input to the first output, the second instance of the input to the second output, and the first set of attributes to the second set of attributes.

公开(公告)号：US20190392213A1

公开(公告)日：2019-12-26

申请号：US16360732

申请日：2019-03-21

Applicant: Apple Inc.

Inventor： Daniel Ulbricht ,  Angela Blechschmidt ,  Mohammad Haris Baig ,  Tanmay Batra ,  Eshan Verma ,  Amit Kumar KC

IPC: G06K9/00 ,  G06K9/62 ,  G06K9/72 ,  G06T7/70 ,  G06T19/00

Abstract: In one implementation, a method of generating a plane hypothesis is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method include obtaining a point cloud based on the image of the scene and generating an object classification set based on the image of the scene, each element of the object classification set including a respective plurality of pixels classified as a respective object in the scene. The method includes generating a plane hypothesis based on the point cloud and the object classification set.