Method and device for tailoring a synthesized reality experience to a physical setting

    公开(公告)号:US11423604B2

    公开(公告)日:2022-08-23

    申请号:US17397437

    申请日:2021-08-09

    Applicant: Apple Inc.

    Abstract: In one implementation, a method includes: obtaining locality data characterizing objects and relative spatial information of a volumetric region around a user; synthesizing a mesh map of the volumetric region based on the locality data; selecting synthesized reality (SR) content based on the mesh map, wherein the SR content satisfies a dimensional variance threshold relative to one or more portions of the mesh map; compositing at least a portion of the SR content with the mesh map in order to generate composite SR content; and presenting the composite SR content to the user in order to occlude at least a portion of a visual presentation of the volumetric region. In some implementations, the SR content is adapted to fit the one or more portions of the mesh map. In some implementations, the SR content is updated as the user location changes or the user interacts with the SR content.

    Method and device for tailoring a synthesized reality experience to a physical setting

    公开(公告)号:US11120612B2

    公开(公告)日:2021-09-14

    申请号:US16960163

    申请日:2019-01-18

    Applicant: Apple Inc.

    Abstract: In one implementation, a method includes: obtaining locality data characterizing objects and relative spatial information of a volumetric region around a user; synthesizing a mesh map of the volumetric region based on the locality data; selecting synthesized reality (SR) content based on the mesh map, wherein the SR content satisfies a dimensional variance threshold relative to one or more portions of the mesh map; compositing at least a portion of the SR content with the mesh map in order to generate composite SR content; and presenting the composite SR content to the user in order to occlude at least a portion of a visual presentation of the volumetric region. In some implementations, the SR content is adapted to fit the one or more portions of the mesh map. In some implementations, the SR content is updated as the user location changes or the user interacts with the SR content.

    Temporal Blending of Depth Maps
    4.
    发明公开

    公开(公告)号:US20240265636A1

    公开(公告)日:2024-08-08

    申请号:US18436616

    申请日:2024-02-08

    Applicant: Apple Inc.

    Abstract: In one implementation, a method of generating a depth map is performed by a device including one or more processors and non-transitory memory. The method includes obtaining a first mesh of a physical environment and a second mesh of the physical environment. The method includes rendering a first depth map for an image of the physical environment based on the first mesh and a second depth map for the image of the physical environment based on the second mesh. The method includes generating a blended depth map based on the first depth map, the second depth map, and a difference between a time of the image of the physical environment and a time of the second mesh.

    Temporally Stable Perspective Correction
    5.
    发明公开

    公开(公告)号:US20240078692A1

    公开(公告)日:2024-03-07

    申请号:US18242339

    申请日:2023-09-05

    Applicant: Apple Inc.

    CPC classification number: G06T7/50 G06T17/20

    Abstract: In one implementation, a method of performing perspective correction is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, an image of a physical environment. The method includes obtaining a depth map including a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment, wherein the depth map includes, for a particular pixel at a particular pixel location representing a dynamic object in the physical environment, a particular depth corresponding to a distance between the image sensor and a static object in the physical environment behind the dynamic object. The method includes transforming, using the one or more processors, the image of the physical environment based on the depth map. The method includes displaying, on the display, the transformed image.

    Multi-resolution voxel meshing
    7.
    发明授权

    公开(公告)号:US11328481B2

    公开(公告)日:2022-05-10

    申请号:US17147559

    申请日:2021-01-13

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein generate a mesh representing the surfaces in a physical environment. The mesh is generated using multi-resolution voxels based on detected depth information, e.g., from a depth camera. The techniques may use multiple hash tables to store the multi-resolution voxel data. For example, the hash tables may store each voxel's 3D position and a truncated signed distance field (TSDF) value corresponding to each voxels' distance to a nearest surface. Each of the multiple hash tables may include data corresponding to a different level of resolution and those resolutions may depend upon distance/noise or other factors. For example, voxels close to a depth camera may have a finer resolution and smaller size compared to voxels that are further from the depth camera. Techniques disclosed herein may involve using a meshing algorithm that combines multi-resolution voxel information stored in multiple hash tables to generate a single mesh.

    Reconstructing views of real world 3D scenes

    公开(公告)号:US10984583B2

    公开(公告)日:2021-04-20

    申请号:US16288211

    申请日:2019-02-28

    Applicant: Apple Inc.

    Inventor: Maxime Meilland

    Abstract: Various implementations are disclosed of producing a 3-dimensional model of a scene. Various method, electronic device, or system implementations use RGB-D camera to provide RGB-D video content or periodic aligned RGB images and depth images to localize camera spatial position(s) defined in a three dimensional (3D) coordinate system or reconstruct a 3D virtual representation of a current camera frame in the 3D coordinate system, each in real time.

    RECONSTRUCTING VIEWS OF REAL WORLD 3D SCENES

    公开(公告)号:US20190304170A1

    公开(公告)日:2019-10-03

    申请号:US16288211

    申请日:2019-02-28

    Applicant: Apple Inc.

    Inventor: Maxime Meilland

    Abstract: Various implementations are disclosed of producing a 3-dimensional model of a scene. Various method, electronic device, or system implementations use RGB-D camera to provide RGB-D video content or periodic aligned RGB images and depth images to localize camera spatial position(s) defined in a three dimensional (3D) coordinate system or reconstruct a 3D virtual representation of a current camera frame in the 3D coordinate system, each in real time.

    MULTI-RESOLUTION VOXEL MESHING
    10.
    发明申请

    公开(公告)号:US20210225074A1

    公开(公告)日:2021-07-22

    申请号:US17147559

    申请日:2021-01-13

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein generate a mesh representing the surfaces in a physical environment. The mesh is generated using multi-resolution voxels based on detected depth information, e.g., from a depth camera. The techniques may use multiple hash tables to store the multi-resolution voxel data. For example, the hash tables may store each voxel's 3D position and a truncated signed distance field (TSDF) value corresponding to each voxels' distance to a nearest surface. Each of the multiple hash tables may include data corresponding to a different level of resolution and those resolutions may depend upon distance/noise or other factors. For example, voxels close to a depth camera may have a finer resolution and smaller size compared to voxels that are further from the depth camera. Techniques disclosed herein may involve using a meshing algorithm that combines multi-resolution voxel information stored in multiple hash tables to generate a single mesh.

Patent Agency Ranking