公开(公告)号：US11442539B2

公开(公告)日：2022-09-13

申请号：US16963633

申请日：2019-01-23

Applicant: APPLE INC.

Inventor： Thomas Gebauer ,  Raffi Bedikian

IPC: G06F3/01 ,  G06K9/62 ,  G06T3/40 ,  G06V10/141 ,  G06V40/18

Abstract: One implementation involves a device receiving a stream of pixel events output by an event camera. The device derives an input image by accumulating pixel events for multiple event camera pixels. The device generates a gaze characteristic using the derived input image as input to a neural network trained to determine the gaze characteristic. The neural network is configured in multiple stages. The first stage of the neural network is configured to determine an initial gaze characteristic, e.g., an initial pupil center, using reduced resolution input(s). The second stage of the neural network is configured to determine adjustments to the initial gaze characteristic using location-focused input(s), e.g., using only a small input image centered around the initial pupil center. The determinations at each stage are thus efficiently made using relatively compact neural network configurations. The device tracks a gaze of the eye based on the gaze characteristic.

公开(公告)号：US11861873B2

公开(公告)日：2024-01-02

申请号：US17881692

申请日：2022-08-05

Applicant: Apple Inc.

Inventor： Thomas Gebauer ,  Raffi Bedikian

IPC: G06V10/141 ,  G06F3/01 ,  G06T3/40 ,  G06V40/18 ,  G06F18/214 ,  G06F18/2413 ,  G06V10/764 ,  G06V10/82 ,  G06V10/44

CPC classification number: G06V10/141 ,  G06F3/013 ,  G06F18/214 ,  G06F18/2413 ,  G06T3/4046 ,  G06V10/454 ,  G06V10/764 ,  G06V10/82 ,  G06V40/193

Abstract: One implementation involves a device receiving a stream of pixel events output by an event camera. The device derives an input image by accumulating pixel events for multiple event camera pixels. The device generates a gaze characteristic using the derived input image as input to a neural network trained to determine the gaze characteristic. The neural network is configured in multiple stages. The first stage of the neural network is configured to determine an initial gaze characteristic, e.g., an initial pupil center, using reduced resolution input(s). The second stage of the neural network is configured to determine adjustments to the initial gaze characteristic using location-focused input(s), e.g., using only a small input image centered around the initial pupil center. The determinations at each stage are thus efficiently made using relatively compact neural network configurations. The device tracks a gaze of the eye based on the gaze characteristic.