公开(公告)号：US20180113508A1

公开(公告)日：2018-04-26

申请号：US15788725

申请日：2017-10-19

Applicant: Apple Inc.

Inventor： Kathrin Berkner-Cieslicki ,  Ricardo J. Motta ,  Se Hoon Lim ,  Minwoong Kim ,  Kenichi Saito ,  Branko Petljanski ,  Jason C. Sauers ,  Yoshikazu Shinohara

IPC: G06F3/01 ,  G06T19/00 ,  H04N5/33 ,  G06T7/292

CPC classification number: G06F3/013 ,  G02B27/0172 ,  G02B2027/0187 ,  G06T7/292 ,  G06T19/006 ,  H04N5/33

Abstract: An eye tracking system for detecting position and movements of a user's eyes in a head-mounted display (HMD). The eye tracking system may include at least one near-IR (NIR) eye tracking camera positioned at each side of the user's face and pointed towards eye-facing surfaces of eyepieces of the HMD, an illumination source that emits NIR light towards the user's eyes, and hot mirrors located between the eye-facing surfaces of the eyepieces and the user's eyes. The hot mirrors reflect at least a portion of NIR light, while allowing visible light to pass. The cameras capture images of the user's eyes reflected by the hot mirrors. Positioning the hot mirrors at or near the eye-facing surfaces of the eyepieces allows the cameras to be placed at the sides of the user's face without having to image through the eyepieces.

公开(公告)号：US12112449B2

公开(公告)日：2024-10-08

申请号：US18337769

申请日：2023-06-20

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/18 ,  G06T7/50 ,  G06T7/521 ,  G06T7/593 ,  G06T7/70 ,  G06V20/10 ,  H04N5/262 ,  H04N9/73 ,  H04N23/63 ,  H04N23/67 ,  H04N23/90

CPC classification number: G06T3/18 ,  G06T7/50 ,  G06T7/70 ,  G06V20/10 ,  H04N5/2621 ,  H04N9/73 ,  H04N23/63 ,  H04N23/90 ,  G06T7/521 ,  G06T7/593 ,  G06T2207/20164 ,  G06T2207/30201 ,  H04N23/67

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US11250541B2

公开(公告)日：2022-02-15

申请号：US16125363

申请日：2018-09-07

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/00 ,  G06T7/50 ,  G06T7/70 ,  G06K9/00 ,  H04N5/232 ,  H04N5/247 ,  H04N5/262 ,  H04N9/73 ,  G06T7/521 ,  G06T7/593

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US20190206120A1

公开(公告)日：2019-07-04

申请号：US16291590

申请日：2019-03-04

Applicant: Apple Inc.

Inventor： Ricardo Motta ,  Lynn R. Youngs ,  Minwoong Kim

IPC: G06T15/50 ,  G06T15/60 ,  G06T15/20 ,  G06F3/0346 ,  G06K9/00

Abstract: Techniques are disclosed for displaying a graphical element in a manner that simulates three-dimensional (3D) visibility (including parallax and shadowing). More particularly, a number of images, each captured with a known spatial relationship to a target 3D object, may be used to construct a lighting model of the target object. In one embodiment, for example, polynomial texture maps (PTM) using spherical or hemispherical harmonics may be used to do this. Using PTM techniques a relatively small number of basis images may be identified. When the target object is to be displayed, orientation information may be used to generate a combination of the basis images so as to simulate the 3D presentation of the target object.

公开(公告)号：US20230334617A1

公开(公告)日：2023-10-19

申请号：US18337769

申请日：2023-06-20

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/00 ,  G06V20/10 ,  H04N23/63 ,  H04N23/90 ,  G06T7/50 ,  G06T7/70 ,  H04N5/262 ,  H04N9/73

CPC classification number: G06T3/0093 ,  G06V20/10 ,  H04N23/63 ,  H04N23/90 ,  G06T7/50 ,  G06T7/70 ,  H04N5/2621 ,  H04N9/73 ,  H04N23/67

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US20220172319A1

公开(公告)日：2022-06-02

申请号：US17651001

申请日：2022-02-14

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/00 ,  G06V20/10 ,  G06T7/50 ,  G06T7/70 ,  H04N5/232 ,  H04N5/247 ,  H04N5/262 ,  H04N9/73

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US20250104183A1

公开(公告)日：2025-03-27

申请号：US18907344

申请日：2024-10-04

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/18 ,  G06T7/50 ,  G06T7/521 ,  G06T7/593 ,  G06T7/70 ,  G06V20/10 ,  H04N5/262 ,  H04N9/73 ,  H04N23/63 ,  H04N23/67 ,  H04N23/90

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US11720996B2

公开(公告)日：2023-08-08

申请号：US17651001

申请日：2022-02-14

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/00 ,  G06V20/10 ,  H04N23/63 ,  H04N23/90 ,  G06T7/50 ,  G06T7/70 ,  H04N5/262 ,  H04N9/73 ,  H04N23/67 ,  G06T7/521 ,  G06T7/593

CPC classification number: G06T3/0093 ,  G06T7/50 ,  G06T7/70 ,  G06V20/10 ,  H04N5/2621 ,  H04N9/73 ,  H04N23/63 ,  H04N23/90 ,  G06T7/521 ,  G06T7/593 ,  G06T2207/20164 ,  G06T2207/30201 ,  H04N23/67

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US10748331B2

公开(公告)日：2020-08-18

申请号：US16291590

申请日：2019-03-04

Applicant: Apple Inc.

Inventor： Ricardo Motta ,  Lynn R. Youngs ,  Minwoong Kim

IPC: G06T15/50 ,  G06T15/20 ,  G06F3/0346 ,  G06K9/00 ,  G06T15/60

Abstract: Techniques are disclosed for displaying a graphical element in a manner that simulates three-dimensional (3D) visibility (including parallax and shadowing). More particularly, a number of images, each captured with a known spatial relationship to a target 3D object, may be used to construct a lighting model of the target object. In one embodiment, for example, polynomial texture maps (PTM) using spherical or hemispherical harmonics may be used to do this. Using PTM techniques a relatively small number of basis images may be identified. When the target object is to be displayed, orientation information may be used to generate a combination of the basis images so as to simulate the 3D presentation of the target object.

公开(公告)号：US10262452B2

公开(公告)日：2019-04-16

申请号：US15274284

申请日：2016-09-23

Applicant: Apple Inc.

Inventor： Ricardo Motta ,  Lynn R. Youngs ,  Minwoong Kim

IPC: G06T15/60 ,  G06T15/50 ,  G06F3/0346 ,  G06K9/00 ,  G06T15/20

Abstract: Techniques are disclosed for displaying a graphical element in a manner that simulates three-dimensional (3D) visibility (including parallax and shadowing). More particularly, a number of images, each captured with a known spatial relationship to a target 3D object, may be used to construct a lighting model of the target object. In one embodiment, for example, polynomial texture maps (PTM) using spherical or hemispherical harmonics may be used to do this. Using PTM techniques a relatively small number of basis images may be identified. When the target object is to be displayed, orientation information may be used to generate a combination of the basis images so as to simulate the 3D presentation of the target object.