-
公开(公告)号:US11238604B1
公开(公告)日:2022-02-01
申请号:US16789788
申请日:2020-02-13
Applicant: Apple Inc.
Inventor: Mohammad Haris Baig , Daniel Ulbricht
Abstract: A system and techniques that use one or more machine learning models to predict a dense depth map (e.g., of depth values for all pixels or at least more pixels than a sparse estimation source (e.g., SLAM)). In some implementations, the machine learning model includes two sub models (e.g., neural networks). The first machine learning model predicts computer vision data such as semantic labels and surface normal directions from an input image. This computer vision data will be used to add to or otherwise improve sparse depth data. Specifically, a second machine learning model takes the semantic labels and surface normal directions from and sparse depth data (e.g., 3D points) from a sparse point estimation source (e.g., SLAM) as inputs and outputs a depth map. The output depth map effectively densities the initial depth data (e.g., from SLAM) by providing depth data for additional pixels of the image.
-
公开(公告)号:US10824864B2
公开(公告)日:2020-11-03
申请号:US16360732
申请日:2019-03-21
Applicant: Apple Inc.
Inventor: Daniel Ulbricht , Angela Blechschmidt , Mohammad Haris Baig , Tanmay Batra , Eshan Verma , Amit Kumar KC
Abstract: In one implementation, a method of generating a plane hypothesis is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method include obtaining a point cloud based on the image of the scene and generating an object classification set based on the image of the scene, each element of the object classification set including a respective plurality of pixels classified as a respective object in the scene. The method includes generating a plane hypothesis based on the point cloud and the object classification set.
-
公开(公告)号:US11315278B1
公开(公告)日:2022-04-26
申请号:US16580181
申请日:2019-09-24
Applicant: Apple Inc.
Inventor: Daniel Ulbricht , Amit Kumar K C , Angela Blechschmidt , Chen-Yu Lee , Eshan Verma , Mohammad Haris Baig , Tanmay Batra
Abstract: In one implementation, a method of estimating the orientation of an object in an image is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels at a respective plurality of pixel locations and having a respective plurality of pixel values. The method includes determining a first set of pixels locations corresponding to a 2D boundary surrounding an object represented in the image and determining, based on the first set of pixel locations, a second set of pixel locations corresponding to a 3D boundary surrounding the object.
-
公开(公告)号:US20220114796A1
公开(公告)日:2022-04-14
申请号:US17557805
申请日:2021-12-21
Applicant: Apple Inc.
Inventor: Daniel Ulbricht , Amit Kumar K C , Angela Blechschmidt , Chen-Yu Lee , Eshan Verma , Mohammad Haris Baig , Tanmay Batra
IPC: G06T19/00 , G06F3/01 , A63F13/825 , G02B27/01 , A63F13/212 , G06F3/03
Abstract: In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.
-
公开(公告)号:US20210134067A1
公开(公告)日:2021-05-06
申请号:US17149949
申请日:2021-01-15
Applicant: Apple Inc.
Inventor: Daniel Ulbricht , Amit Kumar K C , Angela Blechschmidt , Chen-Yu Lee , Eshan Verma , Mohammad Haris Baig , Tanmay Batra
IPC: G06T19/00 , G06F3/01 , A63F13/825 , G02B27/01 , A63F13/212 , G06F3/03
Abstract: In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.
-
公开(公告)号:US10832487B1
公开(公告)日:2020-11-10
申请号:US16580172
申请日:2019-09-24
Applicant: Apple Inc.
Inventor: Daniel Ulbricht , Amit Kumar K C , Angela Blechschmidt , Chen-Yu Lee , Eshan Verma , Mohammad Haris Baig , Tanmay Batra
Abstract: In one implementation, a method of generating a depth map is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes generating, based on a first image and a second image, a first depth map of the second image. The method includes generating, based on the first depth map of the second image and pixel values of the second image, a second depth map of the second image.
-
公开(公告)号:US12100194B1
公开(公告)日:2024-09-24
申请号:US17356642
申请日:2021-06-24
Applicant: Apple Inc.
Abstract: Various implementations disclosed herein include devices, systems, and methods that enhance an image based on depth and/or other sensor data. For example, an example process may include obtaining an image of a physical environment from a first sensor, each pixel of the image corresponding to an amount of light, obtaining depth data of the physical environment via a second sensor, and enhancing the image based on the depth sensor data.
-
公开(公告)号:US11710283B2
公开(公告)日:2023-07-25
申请号:US17508010
申请日:2021-10-22
Applicant: Apple Inc.
Inventor: Eshan Verma , Daniel Ulbricht , Angela Blechschmidt , Mohammad Haris Baig , Chen-Yu Lee , Tanmay Batra
IPC: G06T19/00
CPC classification number: G06T19/006
Abstract: Various implementations disclosed herein include devices, systems, and methods that enable faster and more efficient real-time physical object recognition, information retrieval, and updating of a CGR environment. In some implementations, the CGR environment is provided at a first device based on a classification of the physical object, image or video data including the physical object is transmitted by the first device to a second device, and the CGR environment is updated by the first device based on a response associated with the physical object received from the second device.
-
公开(公告)号:US11295529B2
公开(公告)日:2022-04-05
申请号:US17149949
申请日:2021-01-15
Applicant: Apple Inc.
Inventor: Daniel Ulbricht , Amit Kumar K C , Angela Blechschmidt , Chen-Yu Lee , Eshan Verma , Mohammad Haris Baig , Tanmay Batra
IPC: G06T19/00 , G06F3/01 , A63F13/825 , G02B27/01 , A63F13/212 , G06F3/03
Abstract: In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.
-
公开(公告)号:US11132546B2
公开(公告)日:2021-09-28
申请号:US17032213
申请日:2020-09-25
Applicant: Apple Inc.
Inventor: Daniel Ulbricht , Angela Blechschmidt , Mohammad Haris Baig , Tanmay Batra , Eshan Verma , Amit Kumar KC
Abstract: In one implementation, a method of generating a plane hypothesis is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method include obtaining a point cloud based on the image of the scene and generating an object classification set based on the image of the scene, each element of the object classification set including a respective plurality of pixels classified as a respective object in the scene. The method includes generating a plane hypothesis based on the point cloud and the object classification set.
-
-
-
-
-
-
-
-
-