Object Detection Using Multiple Three Dimensional Scans

    公开(公告)号:US20200020118A1

    公开(公告)日:2020-01-16

    申请号:US16458425

    申请日:2019-07-01

    Applicant: Apple Inc.

    Abstract: One exemplary implementation facilitates object detection using multiple scans of an object in different conditions. For example, a first scan of the object can be created by capturing images of the object by moving an image sensor on a first path in a first condition, e.g., bright lighting. A second scan of the object can then be created by capturing additional images of the object by moving the image sensor on a second path in a second condition, e.g., dim lighting. Implementations determine a transform that associates the scan data from these multiple scans to one another and use the transforms to generate a 3D model of the object in a single coordinate system. Augmented content can be positioned relative to that object in the single coordinate system and thus will be displayed in the appropriate location regardless of the condition in which the physical object is later detected.

    Method of providing image feature descriptors

    公开(公告)号:US10528847B2

    公开(公告)日:2020-01-07

    申请号:US16531678

    申请日:2019-08-05

    Applicant: Apple Inc.

    Abstract: A method of providing a set of feature descriptors configured to be used in matching an object in an image of a camera is provided. The method includes: a) providing at least two images of a first object; b) extracting in at least two of the images at least one feature from the respective image, c) providing at least one descriptor for an extracted feature, and storing the descriptors; d) matching descriptors in the first set of descriptors; e) computing a score parameter based on the result of the matching process; f) selecting at least one descriptor based on its score parameter; g) adding the selected descriptor(s) to a second set of descriptors; and h) updating the score parameter of descriptors in the first set based on a selection process and to the result of the matching process.

    Visual marker
    23.
    发明授权

    公开(公告)号:US12093763B2

    公开(公告)日:2024-09-17

    申请号:US18010578

    申请日:2021-06-15

    Applicant: APPLE INC.

    CPC classification number: G06K19/06168

    Abstract: Various implementations disclosed herein include devices, systems, and methods that provide a visual marker including a plurality of markings arranged in a corresponding plurality of shapes. In some implementations, each marking is formed of a set of sub-markings separated by gaps and arranged according to a respective shape, and the gaps of the plurality of markings are configured to encode data and indicate orientation of the visual marker. In some implementations, the plurality of markings are arranged in a plurality of concentric rings of increasing size. In some implementations, the orientation is encoded in a first set of gaps and data in a second set of gaps of the gaps in the plurality of markings.

    Image-based detection of surfaces that provide specular reflections and reflection modification

    公开(公告)号:US11462000B2

    公开(公告)日:2022-10-04

    申请号:US17001788

    申请日:2020-08-25

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein include devices, systems, and methods that detect surfaces and reflections in such surfaces. Some implementations involve providing a CGR environment that includes virtual content that replaces the appearance of a user or the user's device in a mirror or other surface providing a reflection. For example, a CGR environment may be modified to include a reflection of the user that does not include the device that the user is holding or wearing. In another example, the CGR environment is modified so that virtual content, such as a newer version of the electronic device or a virtual wand, replaces the electronic device in the reflection. In another example, the CGR environment is modified so that virtual content, such as a user avatar, replaces the user in the reflection.

    Adaptive image warping based on object and distance information

    公开(公告)号:US11373271B1

    公开(公告)日:2022-06-28

    申请号:US17203069

    申请日:2021-03-16

    Applicant: Apple Inc.

    Abstract: A method includes obtaining an image via an image sensor, and identifying, within the image, a physical object represented by a portion of the image. The method includes determining, based on the image, a visual feature characterizing the physical object. The method includes warping, based on the visual feature satisfying a first feature criterion, the portion of the image according to a first warping function that is based on the first feature criterion and a distance between the electronic device and a reference point. The method includes warping, based on the visual feature satisfying a second feature criterion that is different from the first feature criterion, the portion of the image according to a second warping function that is based on the second feature criterion and the distance between the electronic device and the reference point.

    Mirror accessory for camera based touch detection

    公开(公告)号:US11281337B1

    公开(公告)日:2022-03-22

    申请号:US17021448

    申请日:2020-09-15

    Applicant: Apple Inc.

    Abstract: Touch detection using a mirror accessory may include obtaining first image data comprising a touching object and a target surface, wherein the first image data captures a scene comprising the touching object and the target surface from the viewpoint of a camera, obtaining second image data comprising the touching object and the target surface, wherein the second image data captures the touching object and the target surface as a reflection in a mirror that is separate from the target surface, determining a pose of the mirror in the scene, determining a pose of the touching object in the scene based on the first image data, the second image data, and the pose of the mirror, and estimating a touch status between the touching object and the target surface based on the determined pose of the touching object and a pose of the target surface.

    Method and device for presenting synthesized reality content in association with recognized objects

    公开(公告)号:US11210520B2

    公开(公告)日:2021-12-28

    申请号:US16959253

    申请日:2019-01-18

    Applicant: Apple Inc.

    Abstract: In one implementation, a method includes: obtaining image data from an image sensor; recognizing a portion of an object within the image data; obtaining synthesized reality (SR) content—such as mixed reality, augmented reality, augmented virtuality, or virtual reality content—associated with the portion of the object; and displaying the SR content in association with the portion of the object. In some implementations, the SR content is dependent on the orientation of an electronic device or the user relative to the object. In some implementations, the SR content is generated based on sensor data associated with the object.

    Localization and mapping using images from multiple devices

    公开(公告)号:US11189054B2

    公开(公告)日:2021-11-30

    申请号:US16554461

    申请日:2019-08-28

    Applicant: Apple Inc.

    Abstract: The present disclosure relates generally to localization and mapping. In some examples, an electronic device obtains first image data and motion data using a motion sensor. The electronic device receives information corresponding to a second electronic device. The electronic device generates a representation of a first pose of the first electronic device using the first image data, the motion data, and the information corresponding to the second electronic device. The electronic device displays, on the display, a virtual object, wherein the displaying of the virtual object is based on the representation of the first pose of the first electronic device.

    Tracking and drift correction
    29.
    发明授权

    公开(公告)号:US11036284B2

    公开(公告)日:2021-06-15

    申请号:US16567970

    申请日:2019-09-11

    Applicant: APPLE INC.

    Abstract: Some implementations provide improved user interfaces for interacting with a virtual environment. The virtual environment is presented by a display of a first device having an image sensor. The first device uses the image sensor to determine a relative position and orientation of a second device based on a marker displayed on a display of the second device. The first device uses the determined relative position of the second device to display a representation of the second device including virtual content in place of the marker.

    AUGMENTED DEVICES
    30.
    发明申请

    公开(公告)号:US20210096638A1

    公开(公告)日:2021-04-01

    申请号:US17019856

    申请日:2020-09-14

    Applicant: Apple Inc.

    Abstract: Implementations use a first device (e.g., an HMD) to provide a CGR environment that augments the input and output capabilities of a second device, e.g., a laptop, smart speaker, etc. In some implementations, the first device communicates with a second device in its proximate physical environment to exchange input or output data. For example, an HMD may capture an image of a physical environment that includes a laptop. The HMD may detect the laptop, send a request the laptop's content, receive content from the laptop (e.g., the content that the laptop is currently displaying and additional content), identify the location of the laptop, and display a virtual object with the received content in the CGR environment on or near the laptop. The size, shape, orientation, or position of the virtual object (e.g., a virtual monitor or monitor extension) may also be configured to provide a better user experience.

Patent Agency Ranking