公开(公告)号：US11710350B2

公开(公告)日：2023-07-25

申请号：US17499205

申请日：2021-10-12

Applicant: Apple Inc.

Inventor： Tom Sengelaub ,  Hua Gao ,  Hao Qin ,  Julia Benndorf

IPC: G06V40/19 ,  G06F3/01 ,  G06T7/73 ,  G06T7/50 ,  G06T7/90 ,  G06T7/70 ,  G06V40/16

CPC classification number: G06V40/19 ,  G06F3/013 ,  G06T7/50 ,  G06T7/70 ,  G06T7/74 ,  G06T7/90 ,  G06V40/165 ,  G06T2200/04 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/10048 ,  G06T2207/30041 ,  G06T2207/30201

Abstract: Some implementations of the disclosure involve, at a device having one or more processors, one or more image sensors, and an illumination source, detecting a first attribute of an eye based on pixel differences associated with different wavelengths of light in a first image of the eye. These implementations next determine a first location associated with the first attribute in a three dimensional (3D) coordinate system based on depth information from a depth sensor. Various implementations detect a second attribute of the eye based on a glint resulting from light of the illumination source reflecting off a cornea of the eye. These implementations next determine a second location associated with the second attribute in the 3D coordinate system based on the depth information from the depth sensor, and determine a gaze direction in the 3D coordinate system based on the first location and the second location.

公开(公告)号：US11170212B2

公开(公告)日：2021-11-09

申请号：US16570389

申请日：2019-09-13

Applicant: Apple Inc.

Inventor： Tom Sengelaub ,  Hua Gao ,  Hao Qin ,  Julia Benndorf

IPC: G06F3/01 ,  G06K9/00 ,  G06T7/73 ,  G06T7/50 ,  G06T7/90

Abstract: Some implementations of the disclosure involve, at a device having one or more processors, one or more image sensors, and an illumination source, detecting a first attribute of an eye based on pixel differences associated with different wavelengths of light in a first image of the eye. These implementations next determine a first location associated with the first attribute in a three dimensional (3D) coordinate system based on depth information from a depth sensor. Various implementations detect a second attribute of the eye based on a glint resulting from light of the illumination source reflecting off a cornea of the eye. These implementations next determine a second location associated with the second attribute in the 3D coordinate system based on the depth information from the depth sensor, and determine a gaze direction in the 3D coordinate system based on the first location and the second location.

公开(公告)号：US20220404916A1

公开(公告)日：2022-12-22

申请号：US17893493

申请日：2022-08-23

Applicant: Apple Inc.

Inventor： Arnd Rose ,  Tom Sengelaub ,  Julia Benndorf ,  Marvin Vogel

IPC: G06F3/01 ,  G02B27/00 ,  G02B27/01 ,  G06T19/00

Abstract: A method is performed at a first device with a non-transitory memory, one or more processors, and a network interface. The method includes storing, in the non-transitory memory, reference data describing at least one reference object. The method includes receiving user data via the network interface at a time after completion of a user session of a user of a second device. The user behavior data includes a user behavior characteristic of the user of the second device at a plurality of times during the user session and respective time stamps indicative of the plurality of times during the user session. The method includes combining, using one or more processors, the user behavior data and the reference data based on the respective time stamps to generate data regarding user behavior during the user session with respect to the at least one reference object.

公开(公告)号：US20220027621A1

公开(公告)日：2022-01-27

申请号：US17499205

申请日：2021-10-12

Applicant: Apple Inc.

Inventor： Tom Sengelaub ,  Hua Gao ,  Hao Qin ,  Julia Benndorf

IPC: G06K9/00 ,  G06F3/01 ,  G06T7/70 ,  G06T7/50

Abstract: Some implementations of the disclosure involve, at a device having one or more processors, one or more image sensors, and an illumination source, detecting a first attribute of an eye based on pixel differences associated with different wavelengths of light in a first image of the eye. These implementations next determine a first location associated with the first attribute in a three dimensional (3D) coordinate system based on depth information from a depth sensor. Various implementations detect a second attribute of the eye based on a glint resulting from light of the illumination source reflecting off a cornea of the eye. These implementations next determine a second location associated with the second attribute in the 3D coordinate system based on the depth information from the depth sensor, and determine a gaze direction in the 3D coordinate system based on the first location and the second location.

公开(公告)号：US20240103618A1

公开(公告)日：2024-03-28

申请号：US18470359

申请日：2023-09-19

Applicant: Apple Inc.

Inventor： Julia Benndorf ,  Qichao Fan ,  Julian K. Shutzberg ,  Paul A. Lacey ,  Hua Gao

IPC: G06F3/01 ,  H04N13/344

CPC classification number: G06F3/013 ,  H04N13/344

Abstract: Methods and apparatus for correcting the gaze direction and the origin (entrance pupil) in gaze tracking systems. During enrollment after an eye model is obtained, the pose of the eye when looking at a target prompt is determined. This information is used to estimate the true visual axis of the eye. The visual axis may then be used to correct the point of view (PoV) with respect to the display during use. If a clip-on lens is present, a corrected gaze axis may be calculated based on the known optical characteristics and pose of the clip-on lens. A clip-on corrected entrance pupil may then be estimated by firing two or more virtual rays through the clip-on lens to determine the intersection between the rays and the corrected gaze axis.