公开(公告)号：US11328481B2

公开(公告)日：2022-05-10

申请号：US17147559

申请日：2021-01-13

Applicant: Apple Inc.

Inventor： Maxime Meilland ,  Andrew Predoehl ,  Kyle L. Simek ,  Ming Chuang ,  Pedro A. Pinies Rodriguez

IPC: G06T17/20 ,  G06T3/40

Abstract: Various implementations disclosed herein generate a mesh representing the surfaces in a physical environment. The mesh is generated using multi-resolution voxels based on detected depth information, e.g., from a depth camera. The techniques may use multiple hash tables to store the multi-resolution voxel data. For example, the hash tables may store each voxel's 3D position and a truncated signed distance field (TSDF) value corresponding to each voxels' distance to a nearest surface. Each of the multiple hash tables may include data corresponding to a different level of resolution and those resolutions may depend upon distance/noise or other factors. For example, voxels close to a depth camera may have a finer resolution and smaller size compared to voxels that are further from the depth camera. Techniques disclosed herein may involve using a meshing algorithm that combines multi-resolution voxel information stored in multiple hash tables to generate a single mesh.

公开(公告)号：US11928779B2

公开(公告)日：2024-03-12

申请号：US17717393

申请日：2022-04-11

Applicant: Apple Inc.

Inventor： Maxime Meilland ,  Andrew Predoehl ,  Kyle L. Simek ,  Ming Chuang ,  Pedro A. Pinies Rodriguez

IPC: G06T17/20

CPC classification number: G06T17/20 ,  G06T3/4007 ,  G06T2200/04

Abstract: Various implementations disclosed herein generate a mesh representing the surfaces in a physical environment. The mesh is generated using multi-resolution voxels based on detected depth information, e.g., from a depth camera. The techniques may use multiple hash tables to store the multi-resolution voxel data. For example, the hash tables may store each voxel's 3D position and a truncated signed distance field (TSDF) value corresponding to each voxels' distance to a nearest surface. Each of the multiple hash tables may include data corresponding to a different level of resolution and those resolutions may depend upon distance/noise or other factors. For example, voxels close to a depth camera may have a finer resolution and smaller size compared to voxels that are further from the depth camera. Techniques disclosed herein may involve using a meshing algorithm that combines multi-resolution voxel information stored in multiple hash tables to generate a single mesh.

公开(公告)号：US11816798B1

公开(公告)日：2023-11-14

申请号：US17195817

申请日：2021-03-09

Applicant: Apple Inc.

Inventor： Baptiste Angles ,  Yu Ping Lin ,  Ming Chuang

IPC: G06T17/20 ,  G06T19/20 ,  G06T17/10

CPC classification number: G06T17/205 ,  G06T17/10 ,  G06T19/20 ,  G06T2219/2004

Abstract: Various implementations disclosed herein include devices, systems, and methods that refine a first 3D surface representation (e.g., a 3D point cloud or a 3D mesh) using a second 3D surface representation that includes a 3D geometric primitive. In some implementations, a first 3D surface representation of a physical environment is obtained including points at 3D locations determined based on data generated by a first sensor. In some implementations, a second 3D surface representation corresponding to at least a portion of the physical environment is obtained that includes at least one 3D geometric primitive. In some implementations, a determination whether to adjust the 3D locations of at least one point of the points of the first 3D surface representation is made based on the 3D geometric primitive, and the 3D locations of the at least one point is adjusted to align with the geometric primitive based on the determination.

公开(公告)号：US20240265636A1

公开(公告)日：2024-08-08

申请号：US18436616

申请日：2024-02-08

Applicant: Apple Inc.

Inventor： Maxime Meilland ,  Mengxin Li ,  Ming Chuang

IPC: G06T17/20 ,  G06F3/01 ,  G06T1/60 ,  G06T19/20

CPC classification number: G06T17/205 ,  G06F3/013 ,  G06T1/60 ,  G06T19/20 ,  G06T2219/2016

Abstract: In one implementation, a method of generating a depth map is performed by a device including one or more processors and non-transitory memory. The method includes obtaining a first mesh of a physical environment and a second mesh of the physical environment. The method includes rendering a first depth map for an image of the physical environment based on the first mesh and a second depth map for the image of the physical environment based on the second mesh. The method includes generating a blended depth map based on the first depth map, the second depth map, and a difference between a time of the image of the physical environment and a time of the second mesh.

公开(公告)号：US20240005511A1

公开(公告)日：2024-01-04

申请号：US18345850

申请日：2023-06-30

Applicant: Apple Inc.

Inventor： Maxime Uzan ,  Edwin Iskandar ,  Ming Chuang ,  Pavel V. Dudrenov

IPC: G06T7/11 ,  G06T15/00 ,  G06T13/00 ,  G06T7/70

CPC classification number: G06T7/11 ,  G06T15/005 ,  G06T2207/20021 ,  G06T7/70 ,  G06T13/00

Abstract: Providing a visual treatment based on proximity to an obstruction includes collecting, by a device, a sensor data for an environment, determining a status for each of a plurality of regions of the environment, where at least one region of the environment is assigned an occupied status, and in accordance with a determination that the device satisfies a predetermined closeness threshold to the at least one region of the environment assigned an occupied status, causing a visual treatment to be rendered by the device, where the visual treatment indicates a location of the at least one region of the environment having an occupied status.

公开(公告)号：US20220237872A1

公开(公告)日：2022-07-28

申请号：US17717393

申请日：2022-04-11

Applicant: Apple Inc.

Inventor： Maxine Meilland ,  Andrew Predoehl ,  Kyle L. Simek ,  Ming Chuang ,  Pedro A. Pinies Rodriguez

IPC: G06T17/20 ,  G06T3/40

Abstract: Various implementations disclosed herein generate a mesh representing the surfaces in a physical environment. The mesh is generated using multi-resolution voxels based on detected depth information, e.g., from a depth camera. The techniques may use multiple hash tables to store the multi-resolution voxel data. For example, the hash tables may store each voxel's 3D position and a truncated signed distance field (TSDF) value corresponding to each voxels' distance to a nearest surface. Each of the multiple hash tables may include data corresponding to a different level of resolution and those resolutions may depend upon distance/noise or other factors. For example, voxels close to a depth camera may have a finer resolution and smaller size compared to voxels that are further from the depth camera. Techniques disclosed herein may involve using a meshing algorithm that combines multi-resolution voxel information stored in multiple hash tables to generate a single mesh.

公开(公告)号：US20240077985A1

公开(公告)日：2024-03-07

申请号：US18339104

申请日：2023-06-21

Applicant: Apple Inc.

Inventor： Divya T. Ramakrishnan ,  Brandon J. Van Ryswyk ,  Reinhard Klapfer ,  Antti P. Saarinen ,  Kyle L. Simek ,  Aitor Aldoma Buchaca ,  Tobias Böttger-Brill ,  Robert Maier ,  Ming Chuang

IPC: G06F3/04815 ,  G01B11/02 ,  G06F3/04842 ,  G06T7/593

CPC classification number: G06F3/04815 ,  G01B11/026 ,  G06F3/04842 ,  G06T7/593

Abstract: An electronic device may include one or more sensors that capture sensor data for a physical environment around the electronic device. The sensor data may be used to determine a scene understanding data set for an extended reality environment including the electronic device. The scene understanding data set may include information such as spatial information, information regarding physical objects in the extended reality environment, and information regarding virtual objects in the extended reality environment. When providing scene understanding data to one or more applications running on the electronic device, spatial and/or temporal restrictions may be applied to the scene understanding data set. Scene understanding data that is associated with locations within a boundary and that is associated with times after a cutoff time may be provided to an application.

公开(公告)号：US20210225074A1

公开(公告)日：2021-07-22

申请号：US17147559

申请日：2021-01-13

Applicant: Apple Inc.

Inventor： Maxime Meilland ,  Andrew Predoehl ,  Kyle L. Simek ,  Ming Chuang ,  Pedro A. Pinies Rodriguez

IPC: G06T17/20 ,  G06T3/40

Abstract: Various implementations disclosed herein generate a mesh representing the surfaces in a physical environment. The mesh is generated using multi-resolution voxels based on detected depth information, e.g., from a depth camera. The techniques may use multiple hash tables to store the multi-resolution voxel data. For example, the hash tables may store each voxel's 3D position and a truncated signed distance field (TSDF) value corresponding to each voxels' distance to a nearest surface. Each of the multiple hash tables may include data corresponding to a different level of resolution and those resolutions may depend upon distance/noise or other factors. For example, voxels close to a depth camera may have a finer resolution and smaller size compared to voxels that are further from the depth camera. Techniques disclosed herein may involve using a meshing algorithm that combines multi-resolution voxel information stored in multiple hash tables to generate a single mesh.