公开(公告)号：US10976814B2

公开(公告)日：2021-04-13

申请号：US16376319

申请日：2019-04-05

Applicant: GOOGLE LLC

Inventor： Idris S. Aleem ,  Andrew S. Logan ,  Mayank Bhargava

IPC: G06F3/01 ,  G06T7/73 ,  G06K9/00 ,  G06T7/246 ,  G02B27/01 ,  G02B27/00

Abstract: A method of tracking an eye of a user includes generating an infrared light, scanning the infrared light over the eye, and detecting reflections of the infrared light from the eye over an eye tracking period. A plurality of glints is identified from the reflections of the infrared light detected. A glint center position of each glint in a glint space is determined and transformed to a gaze position in a display space. At least once during the eye tracking period, an image of the eye is reconstructed from a portion of the reflections of the infrared light detected. A pupil is detected from the image, and a pupil center position is determined. A glint-pupil vector is determined from the pupil center position and the glint center position of at least one glint corresponding in space to the pupil. The glint space is recalibrated based on the glint-pupil vector.

公开(公告)号：US10936056B2

公开(公告)日：2021-03-02

申请号：US16376674

申请日：2019-04-05

Applicant: GOOGLE LLC

Inventor： Idris S. Aleem ,  Andrew S. Logan ,  Mayank Bhargava

IPC: G06F3/01 ,  H04N13/383 ,  G02B27/01

Abstract: A method of tracking an eye of a user includes generating infrared light over an eye tracking period, scanning the infrared light over the eye, and detecting reflections of the infrared light from the eye. Shifts in a position of a wearable heads-up display (WHUD) worn on the head of the user are detected during at least a portion of the eye tracking period. Glints are identified from the detected reflections of the infrared light. A drift in a glint center position of an identified glint relative to a glint space is determined based on a detected shift in position of the WHUD corresponding in space to the identified glint. The glint center position is adjusted to compensate for the drift. The adjusted glint center position is transformed from the glint space to a gaze position in a display space in a field of view of the eye.

公开(公告)号：US12142003B2

公开(公告)日：2024-11-12

申请号：US17238861

申请日：2021-04-23

Applicant: Google LLC

Inventor： Idris Syed Aleem ,  Rees Anwyl Samuel Simmons ,  Sushant Umesh Kulkarni ,  Ahmed Gawish ,  Mayank Bhargava

IPC: G06T7/73 ,  G06K19/06 ,  G06V40/16

Abstract: A system and method of detecting display fit measurements and/or ophthalmic measurements for a head mounted wearable computing device including a display device is provided. An image of a fitting frame worn by a user of the computing device is captured by the user, through an application running on the computing device. One or more visual markers each including a distinct pattern are detected in the image including the fitting frame. A model of the fitting frame, and configuration information associated with the fitting frame, are determined based on the detection of the pattern. A three-dimensional pose of the fitting frame is determined based on the detected visual marker(s) and patterns, and the configuration information associated with the fitting frame. The display device of the head mounted wearable computing device can then be configured based on the three-dimensional pose of the fitting frame as captured in the image.

公开(公告)号：US20230410355A1

公开(公告)日：2023-12-21

申请号：US17807230

申请日：2022-06-16

Applicant: GOOGLE LLC

Inventor： Idris Syed Aleem ,  Mayank Bhargava ,  Sushant Umesh Kulkarni ,  Ahmed Gawish ,  Rees Anwyl Samuel Simmons

IPC: G06T7/73 ,  G06V40/16 ,  G06T7/60 ,  G02B27/01

CPC classification number: G06T7/73 ,  G06V40/165 ,  G06T7/60 ,  G02B27/0172 ,  G06T2207/20081 ,  G06T2207/30201 ,  G06T2207/20076 ,  G02B2027/0138 ,  G02B2027/014 ,  G02B2027/0178

Abstract: A system and method of predicting the fitting of a head mounted wearable device for a user is provided. The system detects fixed facial landmarks in image data captured by a computing device of the user. Measurements associated with the detected fixed facial landmarks may be processed by one or more machine learning models and associated algorithm(s) to predict the probability of fitting of a head mounted wearable device on the head of the user.

公开(公告)号：US11619992B2

公开(公告)日：2023-04-04

申请号：US17224823

申请日：2021-04-07

Applicant: GOOGLE LLC

Inventor： Idris S. Aleem ,  Andrew S. Logan ,  Mayank Bhargava

IPC: G06F3/01 ,  G06T7/73 ,  G06T7/246 ,  G02B27/01 ,  G02B27/00 ,  G06V40/19

Abstract: A method of tracking an eye of a user includes generating an infrared light, scanning the infrared light over the eye, and detecting reflections of the infrared light from the eye over an eye tracking period. A plurality of glints is identified from the reflections of the infrared light detected. A glint center position of each glint in a glint space is determined and transformed to a gaze position in a display space. At least once during the eye tracking period, an image of the eye is reconstructed from a portion of the reflections of the infrared light detected. A pupil is detected from the image, and a pupil center position is determined. A glint-pupil vector is determined from the pupil center position and the glint center position of at least one glint corresponding in space to the pupil. The glint space is recalibrated based on the glint-pupil vector.

公开(公告)号：US20230046950A1

公开(公告)日：2023-02-16

申请号：US17444963

申请日：2021-08-12

Applicant: GOOGLE LLC

Inventor： Rees Anwyl Samuel Simmons ,  Idris Syed Aleem ,  Suhant Umesh Kulkarni ,  Mayank Bhargava ,  Ahmed Gawish

IPC: G02C13/00 ,  G02B27/01 ,  G02C7/02 ,  G06T7/73 ,  G06K9/00

Abstract: A system and method of detecting display fit measurements and/or ophthalmic measurements for a head mounted wearable computing device including a display device is provided. An image of a fitting frame worn by a user of the computing device is captured by the user, through an application running on the computing device. One or more keypoints and/or features and/or landmarks are detected in the image including the fitting frame. A three-dimensional pose of the fitting frame is determined based on the detected keypoints and/or features and/or landmarks, and configuration information associated with the fitting frame. The display device of the head mounted wearable computing device can then be configured based on the three-dimensional pose of the fitting frame as captured in the image.

公开(公告)号：US20220343534A1

公开(公告)日：2022-10-27

申请号：US17238861

申请日：2021-04-23

Applicant: Google LLC

Inventor： Idris Syed Aleem ,  Rees Anwyl Samuel Simmons ,  Sushant Umesh Kulkarni ,  Ahmed Gawish ,  Mayank Bhargava

IPC: G06T7/73 ,  G06K9/00 ,  G06K19/06

Abstract: A system and method of detecting display fit measurements and/or ophthalmic measurements for a head mounted wearable computing device including a display device is provided. An image of a fitting frame worn by a user of the computing device is captured by the user, through an application running on the computing device. One or more visual markers each including a distinct pattern are detected in the image including the fitting frame. A model of the fitting frame, and configuration information associated with the fitting frame, are determined based on the detection of the pattern. A three-dimensional pose of the fitting frame is determined based on the detected visual marker(s) and patterns, and the configuration information associated with the fitting frame. The display device of the head mounted wearable computing device can then be configured based on the three-dimensional pose of the fitting frame as captured in the image.

公开(公告)号：US20210223863A1

公开(公告)日：2021-07-22

申请号：US17224823

申请日：2021-04-07

Applicant: GOOGLE LLC

Inventor： Idris S. Aleem ,  Andrew S. Logan ,  Mayank Bhargava

IPC: G06F3/01 ,  G06T7/73 ,  G06K9/00 ,  G06T7/246 ,  G02B27/01 ,  G02B27/00

Abstract: A method of tracking an eye of a user includes generating an infrared light, scanning the infrared light over the eye, and detecting reflections of the infrared light from the eye over an eye tracking period. A plurality of glints is identified from the reflections of the infrared light detected. A glint center position of each glint in a glint space is determined and transformed to a gaze position in a display space. At least once during the eye tracking period, an image of the eye is reconstructed from a portion of the reflections of the infrared light detected. A pupil is detected from the image, and a pupil center position is determined. A glint-pupil vector is determined from the pupil center position and the glint center position of at least one glint corresponding in space to the pupil. The glint space is recalibrated based on the glint-pupil vector.

公开(公告)号：US20240249477A1

公开(公告)日：2024-07-25

申请号：US18157498

申请日：2023-01-20

Applicant: GOOGLE LLC

Inventor： Idris Syed Aleem ,  Mayank Bhargava

IPC: G06T19/00 ,  G02C13/00 ,  G06T7/50 ,  G06T7/60 ,  G06T7/73

CPC classification number: G06T19/00 ,  G02C13/003 ,  G06T7/50 ,  G06T7/60 ,  G06T7/74 ,  G06T2200/08 ,  G06T2207/30201

Abstract: A system and method of predicting fit of a wearable device from image data obtained by a computing device together with position and orientation of the computing device is provided. The system and method may include capturing a series of frames of image data, and detecting one or more fixed features in the series of frames of image data. Position and orientation data associated with the capture of the image data is combined with the position data related to the one or more fixed features, to extract depth data from the series of frames of image data. A three-dimensional model of is generated based on the extracted depth data. The three-dimensional model is processed by a simulator and/or a machine learning model to predict fit of the wearable device for the user.

公开(公告)号：US20240242441A1

公开(公告)日：2024-07-18

申请号：US18156156

申请日：2023-01-18

Applicant: GOOGLE LLC

Inventor： Idris Syed Aleem ,  Mayank Bhargava ,  Sushant Umesh Kulkarni

IPC: G06T19/00 ,  G01B11/00 ,  G01B11/14 ,  G01P13/00 ,  G02C7/02 ,  G06T7/246 ,  G06T7/579 ,  G06T7/60 ,  G06T17/00

CPC classification number: G06T19/00 ,  G01B11/002 ,  G01B11/14 ,  G01P13/00 ,  G02C7/027 ,  G06T7/248 ,  G06T7/579 ,  G06T7/60 ,  G06T17/00 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/30041 ,  G06T2207/30201

Abstract: A system and method of predicting fit of a wearable device from image data obtained by a computing device together with position and orientation of the computing device is provided. The system and method may include capturing a series of frames of image data, and detecting one or more fixed features in the series of frames of image data. Position and orientation data associated with the capture of the image data is combined with the position data related to the one or more fixed features, to extract depth data from the series of frames of image data. A three-dimensional model is generated based on the extracted depth data. The three-dimensional model and/or key points extracted therefrom, can be processed by a simulator and/or a machine learning model to predict fit of the wearable device for the user.