Heuristic for hand tracking correction modes

    公开(公告)号:US12265665B1

    公开(公告)日:2025-04-01

    申请号:US18371098

    申请日:2023-09-21

    Applicant: Apple Inc.

    Abstract: In some implementations, a method includes: determining a location for virtual content; detecting a user interaction with the virtual content; in response to detecting the user interaction with the virtual content, determining a position of a hand gesture during the user interaction with the virtual content; in accordance with a determination that the position of the hand gesture is within a threshold distance relative to the location of the virtual content, generating corrected hand tracking data associated with the user interaction with the virtual content; and in accordance with a determination that the position of the hand gesture is outside of the threshold distance relative to the location of the virtual content, generating uncorrected hand tracking data associated with the user interaction with the virtual content.

    Spatiotemporal smoothing for improved hand tracking

    公开(公告)号:US12223117B1

    公开(公告)日:2025-02-11

    申请号:US18371111

    申请日:2023-09-21

    Applicant: Apple Inc.

    Abstract: In some implementations, a method includes: obtaining uncorrected hand tracking data; obtaining a depth map associated with a physical environment; identifying a position of a portion of the finger within the physical environment based on the depth map and the uncorrected hand tracking data; performing spatial depth smoothing on a region of the depth map adjacent to the position of the portion of the finger; and generating corrected hand tracking data by performing point of view (POV) correction on the uncorrected hand tracking data based on the spatially depth smoothed region of the depth map adjacent to the portion of the finger.

    Spatiotemporal Smoothing for Improved Hand Tracking

    公开(公告)号:US20250155988A1

    公开(公告)日:2025-05-15

    申请号:US19022868

    申请日:2025-01-15

    Applicant: Apple Inc.

    Abstract: In some implementations, a method includes: obtaining uncorrected hand tracking data; obtaining a depth map associated with a physical environment; identifying a position of a portion of the finger within the physical environment based on the depth map and the uncorrected hand tracking data; performing spatial depth smoothing on a region of the depth map adjacent to the position of the portion of the finger; and generating corrected hand tracking data by performing point of view (POV) correction on the uncorrected hand tracking data based on the spatially depth smoothed region of the depth map adjacent to the portion of the finger.

    Method and device for multi-camera hole filling

    公开(公告)号:US12260493B2

    公开(公告)日:2025-03-25

    申请号:US18089664

    申请日:2022-12-28

    Applicant: Apple Inc.

    Abstract: The method includes: obtaining a first image of an environment from a first image sensor associated with first intrinsic parameters; performing a warping operation on the first image according to perspective offset values to generate a warped first image in order to account for perspective differences between the first image sensor and a user of the electronic device; determining an occlusion mask based on the warped first image that includes a plurality of holes; obtaining a second image of the environment from a second image sensor associated with second intrinsic parameters; normalizing the second image based on a difference between the first and second intrinsic parameters to produce a normalized second image; and filling a first set of one or more holes of the occlusion mask based on the normalized second image to produce a modified first image.

    FOCUS ADJUSTMENTS BASED ON ATTENTION
    5.
    发明公开

    公开(公告)号:US20240211035A1

    公开(公告)日:2024-06-27

    申请号:US18390084

    申请日:2023-12-20

    Applicant: Apple Inc.

    CPC classification number: G06F3/013 G06F1/163 H04N23/62

    Abstract: Various implementations disclosed herein include devices, systems, and methods that adjust a focus of a camera based on a distance associated with a determined user attention. For example, an example process may include obtaining sensor data from one or more sensors in a physical environment. The process may include determining at least one gaze direction of at least one eye based on the sensor data. The process may further include determining a distance associated with user attention based on a convergence determined based on an intersection of gaze directions of the at least one gaze direction, or a distance of an object in a 3D representation of the physical environment based on the at least one gaze direction. The process may further include adjusting a focus of a camera of the one or more sensors based on the distance associated with the user attention.

    Temporally Stable Perspective Correction
    7.
    发明公开

    公开(公告)号:US20240078692A1

    公开(公告)日:2024-03-07

    申请号:US18242339

    申请日:2023-09-05

    Applicant: Apple Inc.

    CPC classification number: G06T7/50 G06T17/20

    Abstract: In one implementation, a method of performing perspective correction is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, an image of a physical environment. The method includes obtaining a depth map including a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment, wherein the depth map includes, for a particular pixel at a particular pixel location representing a dynamic object in the physical environment, a particular depth corresponding to a distance between the image sensor and a static object in the physical environment behind the dynamic object. The method includes transforming, using the one or more processors, the image of the physical environment based on the depth map. The method includes displaying, on the display, the transformed image.

    Perspective Correction with Gravitational Smoothing

    公开(公告)号:US20240078640A1

    公开(公告)日:2024-03-07

    申请号:US18241629

    申请日:2023-09-01

    Applicant: Apple Inc.

    CPC classification number: G06T5/002 G06T5/20 G06T7/50

    Abstract: In one implementation, a method of performing perspective correction is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, an image of a physical environment. The method includes obtaining a depth map including a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment. The method includes smoothing the depth map based on a world-fixed vector. The method includes transforming, using the one or more processors, the image of the physical environment based on the smoothed depth map. The method includes displaying, on the display, the transformed image.

    Method and Device for Multi-Camera Hole Filling

    公开(公告)号:US20230377249A1

    公开(公告)日:2023-11-23

    申请号:US18089664

    申请日:2022-12-28

    Applicant: Apple Inc.

    Abstract: The method includes: obtaining a first image of an environment from a first image sensor associated with first intrinsic parameters; performing a warping operation on the first image according to perspective offset values to generate a warped first image in order to account for perspective differences between the first image sensor and a user of the electronic device; determining an occlusion mask based on the warped first image that includes a plurality of holes; obtaining a second image of the environment from a second image sensor associated with second intrinsic parameters; normalizing the second image based on a difference between the first and second intrinsic parameters to produce a normalized second image; and filling a first set of one or more holes of the occlusion mask based on the normalized second image to produce a modified first image.

    Warped Perspective Correction
    10.
    发明公开

    公开(公告)号:US20240202866A1

    公开(公告)日:2024-06-20

    申请号:US18286522

    申请日:2022-03-29

    Applicant: Apple Inc.

    CPC classification number: G06T3/18 G02B27/017 G06T7/55

    Abstract: In one implementation, a method of performing perspective correction of an image is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, an image of a physical environment. The method includes obtaining a plurality of initial depths respectively associated with a plurality of pixels of the image of the physical environment. The method includes generating a depth map for the image of the physical environment based on the plurality of initial depths and a respective plurality of confidences of the plurality of initial depths. The method includes transforming, using the one or more processors, the image of the physical environment based on the depth map and a difference between a perspective of the image sensor and a perspective of a user. The method includes displaying, on the display, the transformed image.

Patent Agency Ranking