Generating a 3D model of a fingertip for visual touch detection

    公开(公告)号:US11620790B2

    公开(公告)日:2023-04-04

    申请号:US17205830

    申请日:2021-03-18

    申请人: Apple Inc.

    摘要: Generating a 3D model may include determining, based on sensor data from a touch sensor on a first device, a touch event, wherein the touch event comprises a touch on the first device by a touching object. Generating a 3D model may also include, in response to a touch event, obtaining a first image of the touching object by a first camera of the first device, and obtaining, from a second device, a second image of the touching object, wherein the first image of the touching object captures a first view of the touching object, and wherein the second image of the touching object captures a second view of the touching object. A model of the touching object is generated based on the first image and the second image.

    IMU for touch detection
    2.
    发明授权

    公开(公告)号:US11360550B2

    公开(公告)日:2022-06-14

    申请号:US17355320

    申请日:2021-06-23

    申请人: Apple Inc.

    摘要: Touch detection may include determining, based on data from an IMU on a first device that monitors movement of a touching object, a touch event, wherein the touch event indicates contact between the touching object and a surface, obtaining a depth image captured by a second device, comprising a touch image, determining a touch point of the object based on the touch image, and providing a response based on the touch point of the object and the touched surface.

    METHOD AND SYSTEM FOR DETERMINING A POSE OF CAMERA

    公开(公告)号:US20180075620A1

    公开(公告)日:2018-03-15

    申请号:US15807263

    申请日:2017-11-08

    申请人: Apple Inc.

    IPC分类号: G06T7/70

    摘要: The invention is related to a method and system for determining a pose of a first camera, comprising providing or receiving a spatial relationship (Rvc 1) between a visual content displayed on a display device and the first camera, receiving image information associated with an image (B1) of at least part of the displayed visual content captured by a second camera, and determining a pose of the first camera according to the image information associated with the image (B1) and the spatial relationship (Rvcl).

    Method of Image Processing for an Augmented Reality Application

    公开(公告)号:US20170301141A1

    公开(公告)日:2017-10-19

    申请号:US15627097

    申请日:2017-06-19

    申请人: Apple Inc.

    IPC分类号: G06T19/00 G06T7/30

    摘要: An apparatus for and method of image processing in an augmented reality application is provided. The method includes the steps of: providing at least one image of a real environment; performing image processing in an augmented reality application with the at least one image employing visualization of overlaying digital information with visual impressions or the image of the real environment and employing vision-based processing or tracking; and adjusting at least one of a parameter and operating flow of the vision-based processing or tracking depending on at least one of the following: a usage of the image processing, a usage of the visualization, a visually perceivable property of the digital information or the real environment, a property of a display device employed in the visualization, or a manner in which a user is viewing the visualization.

    Estimation of spatial relationships between sensors of a multi-sensor device

    公开(公告)号:US11238616B1

    公开(公告)日:2022-02-01

    申请号:US17099932

    申请日:2020-11-17

    申请人: Apple Inc.

    摘要: In one implementation, a device has a processor, a projector, a first infrared (IR) sensor, a second IR sensor, and instructions stored on a computer-readable medium that are executed by the processor to estimate the sensor-to-sensor extrinsic parameters. The projector projects IR pattern elements onto an environment surface. The first sensor captures a first image including first IR pattern elements corresponding to the projected IR pattern elements and the device estimates 3D positions for first IR pattern elements. The second IR sensor captures a second image including second IR pattern elements corresponding to the projected IR pattern elements and the device matches the first IR pattern elements and the second IR pattern elements. Based on this matching, the device estimates a second extrinsic parameter corresponding to a spatial relationship between the first IR sensor and the second IR sensor.

    Generating a 3D model of a fingertip for visual touch detection

    公开(公告)号:US10970922B2

    公开(公告)日:2021-04-06

    申请号:US16580873

    申请日:2019-09-24

    申请人: Apple Inc.

    摘要: Generating a 3D model may include determining, based on sensor data from a touch sensor on a first device, a touch event, wherein the touch event comprises a touch on the first device by a touching object. Generating a 3D model may also include, in response to a touch event, obtaining a first image of the touching object by a first camera of the first device, and obtaining, from a second device, a second image of the touching object, wherein the first image of the touching object captures a first view of the touching object, and wherein the second image of the touching object captures a second view of the touching object. A model of the touching object is generated based on the first image and the second image.

    Method and system for presenting at least part of an image of a real object in a view of a real environment, and method and system for selecting a subset of a plurality of images

    公开(公告)号:US10659750B2

    公开(公告)日:2020-05-19

    申请号:US15328349

    申请日:2014-07-23

    申请人: Apple Inc.

    摘要: The disclosure relates to a method and system for presenting at least part of an image of a real object in a view of a real environment, comprising providing a first image of at least part of a real object captured by a first camera, determining at least part of the real object in the first image as an object image area, determining a first 3D plane relative to the first camera, the first camera being at a position where the first image is captured, providing at least one image feature related to the real object in the first image, providing at least one first ray passing an optical center of the first camera being at a position where the first image is captured and the at least one image feature, determining, according to a first plane normal direction of the first 3D plane, at least one first angle between the first 3D plane and the at least one first ray, providing a second image of a real environment captured by a second camera, determining a second 3D plane relative to the second camera, the second camera being at a position where the second image is captured, mapping at least part of the object image area to a target space in a view of the real environment captured at a viewpoint, wherein the at least one image feature is mapped to at least one target position in a coordinate system of the view, determining at least one second angle between the second 3D plane and at least one second ray passing the viewpoint of capturing the view and the at least one target position according to a second plane normal direction of the second 3D plane, wherein the at least one image feature is mapped to the at least one target position such that a difference between the at least one first angle and the at least one second angle is minimized.

    Pixel Classification Techniques
    9.
    发明申请

    公开(公告)号:US20180018541A1

    公开(公告)日:2018-01-18

    申请号:US15213018

    申请日:2016-07-18

    申请人: Apple Inc.

    IPC分类号: G06K9/62 H04N5/225

    摘要: Systems, methods, and computer readable media to categorize a pixel (or other element) in an image into one of a number of different categories are described. In general, techniques are disclosed for using properties (e.g., statistics) of the regions being categorized to determine the appropriate size of window around a target pixel (element) and, when necessary, the manner in which the window may be changed if the current size is inappropriate. More particularly, adaptive window size selection techniques are disclosed for use when categorizing an image's pixels into one of two categories (e.g., black or white). Statistics of the selected region may be cascaded to determine whether the current evaluation window is acceptable and, if it is not, an appropriate factor by which to change the currently selected window's size

    Method and System for Vision Based 3D Reconstruction and Object Tracking

    公开(公告)号:US20170365092A1

    公开(公告)日:2017-12-21

    申请号:US15188223

    申请日:2016-06-21

    申请人: Apple Inc.

    发明人: Lejing Wang

    IPC分类号: G06T17/00 G06T19/20 H04N5/247

    摘要: Three dimensional model construction includes obtaining a first image of an object captured by a first camera facing a first direction, obtaining a second image of the object captured by a second camera facing a second direction, wherein the second camera captures the second image of the object as a reflection in a mirror, determining a virtual baseline distance between the first camera and a device including the second camera based on a location of the first camera, a known spatial relationship between the first camera and the device, and a reflected distance of the second camera, and generating at least a partial 3D model of the first object using the first image, the second image, and the virtual baseline distance.