Object Relationship Estimation From A 3D Semantic Mesh

    公开(公告)号:US20210073429A1

    公开(公告)日:2021-03-11

    申请号:US16984406

    申请日:2020-08-04

    Applicant: Apple Inc.

    Abstract: Implementations disclosed herein provide systems and methods that determine relationships between objects based on an original semantic mesh of vertices and faces that represent the 3D geometry of a physical environment. Such an original semantic mesh may be generated and used to provide input to a machine learning model that estimates relationships between the objects in the physical environment. For example, the machine learning model may output a graph of nodes and edges indicating that a vase is on top of a table or that a particular instance of a vase, V1, is on top of a particular instance of a table, T1.

    Temporal and geometric consistency in physical setting understanding

    公开(公告)号:US11610414B1

    公开(公告)日:2023-03-21

    申请号:US16795732

    申请日:2020-02-20

    Applicant: Apple Inc.

    Abstract: A machine learning model is trained and used to perform a computer vision task such as semantic segmentation or normal direction prediction. The model uses a current image of a physical setting and input generated from three dimensional (3D) anchor points that store information determined from prior assessments of the physical setting. The 3D anchor points store previously-determined computer vision task information for the physical setting for particular 3D points locations in a 3D worlds space, e.g., an x, y, z coordinate system that is independent of image capture device pose. For example, 3D anchor points may store previously-determined semantic labels or normal directions for 3D points identified by simultaneous localization and mapping (SLAM) processes. The 3D anchor points are stored and used to generate input for the machine model as the model continues to reason about future images of the physical setting.

    Plane Detection Using Semantic Segmentation

    公开(公告)号:US20210406541A1

    公开(公告)日:2021-12-30

    申请号:US17473469

    申请日:2021-09-13

    Applicant: Apple Inc.

    Abstract: In one implementation, a method of generating a plane hypothesis is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes obtaining an image of a scene including a plurality of pixels. The method includes obtaining a plurality of points of a point cloud based on the image of the scene. The method includes obtaining an object classification set based on the image of the scene. Each element of the object classification set includes a plurality of pixels respectively associated with a corresponding object in the scene. The method includes detecting a plane within the scene by identifying a subset of the plurality of points of the point cloud that correspond to a particular element of the object classification set.

Patent Agency Ranking