公开(公告)号：US20230281846A1

公开(公告)日：2023-09-07

申请号：US18145825

申请日：2022-12-22

Applicant: Apple Inc.

Inventor： Luke A. Pillans ,  Aleksandr M. Movshovich ,  Nicholas C. Soldner ,  Daniel Kurz ,  Arthur Y. Zhang

IPC: G06T7/514 ,  G02B27/01 ,  G06T17/00 ,  G06V10/141 ,  G06V10/60 ,  G06V10/762 ,  H04N5/74 ,  H04N23/21

CPC classification number: G06T7/514 ,  G02B27/0172 ,  G06T17/00 ,  G06V10/141 ,  G06V10/60 ,  G06V10/762 ,  H04N5/74 ,  H04N23/21

Abstract: Methods and apparatus for specular surface mapping in which a camera detects reflections of a light source from a specular surface. The detected light sources may be projected onto a celestial sphere as virtual point sources. True positive observations should be tightly clustered on the celestial sphere; thus, false positives may be identified and removed. Specular surface information may then be determined from clusters of the virtual point sources on the celestial sphere. The clusters of virtual point sources on the celestial sphere may be identified and used to identify a surface as a specular surface. The clusters may also be used to extract other information regarding the specular surface, including but not limited to distance to and extent of the specular surface.

公开(公告)号：US11741624B2

公开(公告)日：2023-08-29

申请号：US17076507

申请日：2020-10-21

Applicant: Apple Inc.

Inventor： Sebastian Knorr ,  Daniel Kurz

IPC: G06T7/579 ,  G06T7/246 ,  G06T7/277

CPC classification number: G06T7/579 ,  G06T7/251 ,  G06T7/277 ,  G06T2207/10016 ,  G06T2207/10028 ,  G06T2207/10048 ,  G06T2207/20076 ,  G06T2207/20092 ,  G06T2207/30201 ,  G06T2207/30244

Abstract: Determining spatial coordinates of a 3D reconstruction includes obtaining, from a first camera system, a first image comprising a first real object, obtaining, from a second camera system, a second image comprising a second real object associated with known geometric properties, wherein the first camera system and the second camera system have a known spatial relationship, and determining a scale of the face based on the second image and the known geometric properties of the at least part of the second real object. Determining the spatial coordinates of the 3D reconstruction also includes determining a pose of the first and second camera systems, and determining the spatial coordinates based on the pose of the first camera system and the scale of the at least part of the second real object.

公开(公告)号：US11636578B1

公开(公告)日：2023-04-25

申请号：US17238285

申请日：2021-04-23

Applicant: Apple Inc.

Inventor： Daniel Kurz ,  Gowri Somanath ,  Tobias Holl

IPC: G06T5/00 ,  G06T3/20

Abstract: Various implementations disclosed herein include devices, systems, and methods that complete content for a missing part of an image of an environment. For example, an example process may include obtaining an image including defined content and missing parts for which content is undefined, determining a spatial image transformation for the image based on the defined content and the missing parts of the image, altering the image by applying the spatial image transformation, and completing the altered image.

公开(公告)号：US11507836B1

公开(公告)日：2022-11-22

申请号：US17122327

申请日：2020-12-15

Applicant: Apple Inc.

Inventor： Daniel Kurz ,  Muhammad Ahmed Riaz

IPC: G06N3/08 ,  G06T7/50 ,  G06K9/62 ,  G06T7/70 ,  G10L15/24

Abstract: Various implementations disclosed herein include devices, systems, and methods that involve federated learning techniques that utilize locally-determined ground truth data that may be used in addition to, or in the alternative to, user-provided ground truth data. Some implementations provide an improved federated learning technique that creates ground truth data on the user device using a second prediction technique that differs from a first prediction technique/model that is being trained. The second prediction technique may be better but may be less suited for real time, general use than the first prediction technique.

公开(公告)号：US11462000B2

公开(公告)日：2022-10-04

申请号：US17001788

申请日：2020-08-25

Applicant: Apple Inc.

Inventor： Peter Meier ,  Daniel Kurz ,  Brian Chris Clark ,  Mohamed Selim Ben Himane

IPC: G06V10/60 ,  G06T7/73 ,  G06V10/145 ,  G06V40/16

Abstract: Various implementations disclosed herein include devices, systems, and methods that detect surfaces and reflections in such surfaces. Some implementations involve providing a CGR environment that includes virtual content that replaces the appearance of a user or the user's device in a mirror or other surface providing a reflection. For example, a CGR environment may be modified to include a reflection of the user that does not include the device that the user is holding or wearing. In another example, the CGR environment is modified so that virtual content, such as a newer version of the electronic device or a virtual wand, replaces the electronic device in the reflection. In another example, the CGR environment is modified so that virtual content, such as a user avatar, replaces the user in the reflection.

公开(公告)号：US11379996B2

公开(公告)日：2022-07-05

申请号：US16761582

申请日：2018-11-13

Applicant: APPLE INC.

Inventor： Peter Kaufmann ,  Daniel Kurz ,  Brian Amberg ,  Yanghai Tsin

IPC: G06T7/246 ,  G06T7/55 ,  G06T7/292 ,  G06T7/73 ,  G06T17/00

Abstract: Various implementations disclosed herein include devices, systems, and methods that use event camera data to track deformable objects such as faces, hands, and other body parts. One exemplary implementation involves receiving a stream of pixel events output by an event camera. The device tracks the deformable object using this data. Various implementations do so by generating a dynamic representation of the object and modifying the dynamic representation of the object in response to obtaining additional pixel events output by the event camera. In some implementations, generating the dynamic representation of the object involves identifying features disposed on the deformable surface of the object using the stream of pixel events. The features are determined by identifying patterns of pixel events. As new event stream data is received, the patterns of pixel events are recognized in the new data and used to modify the dynamic representation of the object.

公开(公告)号：US20200293109A1

公开(公告)日：2020-09-17

申请号：US16888985

申请日：2020-06-01

Applicant: Apple Inc.

Inventor： Sebastian Knorr ,  Daniel Kurz

IPC: G06F3/01 ,  G02B27/00 ,  G06F3/048 ,  G06F3/0481

Abstract: Triggering a state change includes displaying a first version of a series of frames based on a first setup configuration, obtaining a second setup configuration for the series of frames, in response to obtaining a second setup configuration, monitoring for a change in an eye status, and in response to detecting a change in the eye status, displaying a second version of the series of frames based on the second setup configuration.

公开(公告)号：US20200273212A1

公开(公告)日：2020-08-27

申请号：US16740972

申请日：2020-01-13

Applicant: Apple Inc.

Inventor： Daniel Kurz ,  Tobias Holl

IPC: G06T11/00

Abstract: Various implementations disclosed herein render virtual content with noise that is similar to or that otherwise better matches the noise found in the images with which the virtual content is combined. Some implementations involve identifying noise data for an image, creating a parameterized noise model based on the noise data, generating a noise pattern approximating noise of the image or another image using the parameterized noise model, and rendering content that includes the image and virtual content with noise added based on the noise pattern.

公开(公告)号：US10719170B2

公开(公告)日：2020-07-21

申请号：US15274968

申请日：2016-09-23

Applicant: Apple Inc.

Inventor： Daniel Kurz

IPC: G06F3/042 ,  G06F3/01 ,  G06T19/00 ,  H04N5/33

Abstract: The present disclosure is related to a method and device for detecting a touch between at least part of a first object and at least part of a second object, wherein the at least part of the first object has a different temperature than the at least part of the second object. The method includes providing at least one thermal image of a portion of the second object, determining in at least part of the at least one thermal image a pattern which is indicative of a particular value or range of temperature or a particular value or range of temperature change, and using the determined pattern for detecting a touch between the at least part of the first object and the at least part of the second object.

公开(公告)号：US10671159B2

公开(公告)日：2020-06-02

申请号：US15712649

申请日：2017-09-22

Applicant: Apple Inc.

Inventor： Sebastian Knorr ,  Daniel Kurz

IPC: G06F3/048 ,  G06F3/01 ,  G02B27/00 ,  G06F3/0481 ,  G02B27/01

Abstract: Triggering a state change includes displaying a first version of a series of frames based on a first setup configuration, obtaining a second setup configuration for the series of frames, in response to obtaining a second setup configuration, monitoring for a change in an eye status, and in response to detecting a change in the eye status, displaying a second version of the series of frames based on the second setup configuration.