-
1.
公开(公告)号:US20240004476A1
公开(公告)日:2024-01-04
申请号:US17809772
申请日:2022-06-29
申请人: GOOGLE LLC
发明人: Dongeek Shin , Andrea Colaco , Stiven Guillaume Francois Morvan , Adam James Banfield , Shahram Izadi
摘要: Techniques of operating an AR system include determining hand gestures formed by a user based on a sequence of two-dimensional images through skin of the user's wrist acquired from a near-infrared camera. Specifically, an image capture device disposed on a band worn around a user's wrist includes a source of electromagnetic radiation, e.g., light-emitting diodes in the infrared (IR) wavelength band that emit the radiation into the user's wrist and an IR detector which produces the sequence of two-dimensional images of a region within a dermal layer in the user's wrist. From this sequence, gesture detection circuitry determines values of a biological flow metric, e.g., a change in perfusion index (PI) between frames of the sequence, based on a trained model that generates the metric from the sequence. Finally, the gesture detection circuitry maps the values of the biological flow metric to specific hand/finger movements that determine a gesture.
-
公开(公告)号:US11687635B2
公开(公告)日:2023-06-27
申请号:US17439762
申请日:2019-09-25
申请人: Google LLC
发明人: Adarsh Prakash Murthy Kowdle , Ruben Manuel Velarde , Zhijun He , Xu Han , Kourosh Derakshan , Shahram Izadi
CPC分类号: G06F21/32 , G06V10/143 , G06V10/235 , G06V40/166 , H04N5/33 , H04N23/71 , H04N23/72
摘要: This document describes techniques and systems that enable automatic exposure and gain control for face authentication. The techniques and systems include a user device initializing a gain for a near-infrared camera system using a default gain. The user device ascertains patch-mean statistics of one or more regions-of-interest of a most-recently captured image that was captured by the near-infrared camera system. The user device computes an update in the initialized gain to provide an updated gain that is usable to scale the one or more regions-of-interest toward a target mean-luminance value. The user device dampens the updated gain by using hysteresis. Then, the user device sets the initialized gain for the near-infrared camera system to the dampened updated gain.
-
公开(公告)号:US20230154051A1
公开(公告)日:2023-05-18
申请号:US17919460
申请日:2020-04-17
申请人: Google LLC
发明人: Danhang Tang , Saurabh Singh , Cem Keskin , Phillip Andrew Chou , Christian Haene , Mingsong Dou , Sean Ryan Francesco Fanello , Jonathan Taylor , Andrea Tagliasacchi , Philip Lindsley Davidson , Yinda Zhang , Onur Gonen Guleryuz , Shahram Izadi , Sofien Bouaziz
IPC分类号: G06T9/00
摘要: Systems and methods are directed to encoding and/or decoding of the textures/geometry of a three-dimensional volumetric representation. An encoding computing system can obtain voxel blocks from a three-dimensional volumetric representation of an object. The encoding computing system can encode voxel blocks with a machine-learned voxel encoding model to obtain encoded voxel blocks. The encoding computing system can decode the encoded voxel blocks with a machine-learned voxel decoding model to obtain reconstructed voxel blocks. The encoding computing system can generate a reconstructed mesh representation of the object based at least in part on the one or more reconstructed voxel blocks. The encoding computing system can encode textures associated with the voxel blocks according to an encoding scheme and based at least in part on the reconstructed mesh representation of the object to obtain encoded textures.
-
公开(公告)号:US11592908B2
公开(公告)日:2023-02-28
申请号:US17249966
申请日:2021-03-19
申请人: GOOGLE LLC
发明人: Dongeek Shin , Shahram Izadi , David Kim , Sofien Bouaziz , Steven Benjamin Goldberg , Ivan Poupyrev , Shwetak N. Patel
摘要: Techniques of identifying gestures include detecting and classifying inner-wrist muscle motions at a user's wrist using micron-resolution radar sensors. For example, a user of an AR system may wear a band around their wrist. When the user makes a gesture to manipulate a virtual object in the AR system as seen in a head-mounted display (HMD), muscles and ligaments in the user's wrist make small movements on the order of 1-3 mm. The band contains a small radar device that has a transmitter and a number of receivers (e.g., three) of electromagnetic (EM) radiation on a chip (e.g., a Soli chip. This radiation reflects off the wrist muscles and ligaments and is received by the receivers on the chip in the band. The received reflected signal, or signal samples, are then sent to processing circuitry for classification to identify the wrist movement as a gesture.
-
公开(公告)号:US11328486B2
公开(公告)日:2022-05-10
申请号:US16861530
申请日:2020-04-29
申请人: Google LLC
发明人: Anastasia Tkach , Ricardo Martin Brualla , Shahram Izadi , Shuoran Yang , Cem Keskin , Sean Ryan Francesco Fanello , Philip Davidson , Jonathan Taylor , Rohit Pandey , Andrea Tagliasacchi , Pavlo Pidlypenskyi
摘要: A method includes receiving a first image including color data and depth data, determining a viewpoint associated with an augmented reality (AR) and/or virtual reality (VR) display displaying a second image, receiving at least one calibration image including an object in the first image, the object being in a different pose as compared to a pose of the object in the first image, and generating the second image based on the first image, the viewpoint and the at least one calibration image.
-
公开(公告)号:US11145075B2
公开(公告)日:2021-10-12
申请号:US16767401
申请日:2019-10-04
申请人: Google LLC
发明人: Julien Valentin , Onur G. Guleryuz , Mira Leung , Maksym Dzitsiuk , Jose Pascoal , Mirko Schmidt , Christoph Rhemann , Neal Wadhwa , Eric Turner , Sameh Khamis , Adarsh Prakash Murthy Kowdle , Ambrus Csaszar , João Manuel Castro Afonso , Jonathan T. Barron , Michael Schoenberg , Ivan Dryanovski , Vivek Verma , Vladimir Tankovich , Shahram Izadi , Sean Ryan Francesco Fanello , Konstantine Nicholas John Tsotsos
摘要: A handheld user device includes a monocular camera to capture a feed of images of a local scene and a processor to select, from the feed, a keyframe and perform, for a first image from the feed, stereo matching using the first image, the keyframe, and a relative pose based on a pose associated with the first image and a pose associated with the keyframe to generate a sparse disparity map representing disparities between the first image and the keyframe. The processor further is to determine a dense depth map from the disparity map using a bilateral solver algorithm, and process a viewfinder image generated from a second image of the feed with occlusion rendering based on the depth map to incorporate one or more virtual objects into the viewfinder image to generate an AR viewfinder image. Further, the processor is to provide the AR viewfinder image for display.
-
公开(公告)号:US20210264632A1
公开(公告)日:2021-08-26
申请号:US17249095
申请日:2021-02-19
申请人: GOOGLE LLC
发明人: Vladimir Tankovich , Christian Haene , Sean Rayn Francesco Fanello , Yinda Zhang , Shahram Izadi , Sofien Bouaziz , Adarsh Prakash Murthy Kowdle , Sameh Khamis
摘要: According to an aspect, a real-time active stereo system includes a capture system configured to capture stereo data, where the stereo data includes a first input image and a second input image, and a depth sensing computing system configured to predict a depth map. The depth sensing computing system includes a feature extractor configured to extract features from the first and second images at a plurality of resolutions, an initialization engine configured to generate a plurality of depth estimations, where each of the plurality of depth estimations corresponds to a different resolution, and a propagation engine configured to iteratively refine the plurality of depth estimations based on image warping and spatial propagation.
-
8.
公开(公告)号:US20200372284A1
公开(公告)日:2020-11-26
申请号:US16616235
申请日:2019-10-16
申请人: Google LLC
发明人: Christoph Rhemann , Abhimitra Meka , Matthew Whalen , Jessica Lynn Busch , Sofien Bouaziz , Geoffrey Douglas Harvey , Andrea Tagliasacchi , Jonathan Taylor , Paul Debevec , Peter Joseph Denny , Sean Ryan Francesco Fanello , Graham Fyffe , Jason Angelo Dourgarian , Xueming Yu , Adarsh Prakash Murthy Kowdle , Julien Pascal Christophe Valentin , Peter Christopher Lincoln , Rohit Kumar Pandey , Christian Häne , Shahram Izadi
摘要: Methods, systems, and media for relighting images using predicted deep reflectance fields are provided. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample includes (i) a group of one-light-at-a-time (OLAT) images that have each been captured when one light of a plurality of lights arranged on a lighting structure has been activated, (ii) a group of spherical color gradient images that have each been captured when the plurality of lights arranged on the lighting structure have been activated to each emit a particular color, and (iii) a lighting direction, wherein each image in the group of OLAT images and each of the spherical color gradient images are an image of a subject, and wherein the lighting direction indicates a relative orientation of a light to the subject; training a convolutional neural network using the group of training samples, wherein training the convolutional neural network comprises: for each training iteration in a series of training iterations and for each training sample in the group of training samples: generating an output predicted image, wherein the output predicted image is a representation of the subject associated with the training sample with lighting from the lighting direction associated with the training sample; identifying a ground-truth OLAT image included in the group of OLAT images for the training sample that corresponds to the lighting direction for the training sample; calculating a loss that indicates a perceptual difference between the output predicted image and the identified ground-truth OLAT image; and updating parameters of the convolutional neural network based on the calculated loss; identifying a test sample that includes a second group of spherical color gradient images and a second lighting direction; and generating a relit image of the subject included in each of the second group of spherical color gradient images with lighting from the second lighting direction using the trained convolutional neural network.
-
公开(公告)号:US12066282B2
公开(公告)日:2024-08-20
申请号:US17413847
申请日:2020-11-11
申请人: GOOGLE LLC
发明人: Sean Ryan Francesco Fanello , Kaiwen Guo , Peter Christopher Lincoln , Philip Lindsley Davidson , Jessica L. Busch , Xueming Yu , Geoffrey Harvey , Sergio Orts Escolano , Rohit Kumar Pandey , Jason Dourgarian , Danhang Tang , Adarsh Prakash Murthy Kowdle , Emily B. Cooper , Mingsong Dou , Graham Fyffe , Christoph Rhemann , Jonathan James Taylor , Shahram Izadi , Paul Ernest Debevec
IPC分类号: G01B11/25 , G01B11/245 , G06T15/50 , G06T17/20
CPC分类号: G01B11/2513 , G01B11/245 , G06T15/506 , G06T17/205
摘要: A lighting stage includes a plurality of lights that project alternating spherical color gradient illumination patterns onto an object or human performer at a predetermined frequency. The lighting stage also includes a plurality of cameras that capture images of an object or human performer corresponding to the alternating spherical color gradient illumination patterns. The lighting stage also includes a plurality of depth sensors that capture depth maps of the object or human performer at the predetermined frequency. The lighting stage also includes (or is associated with) one or more processors that implement a machine learning algorithm to produce a three-dimensional (3D) model of the object or human performer. The 3D model includes relighting parameters used to relight the 3D model under different lighting conditions.
-
公开(公告)号:US20230393665A1
公开(公告)日:2023-12-07
申请号:US18174358
申请日:2023-02-24
申请人: GOOGLE LLC
发明人: Dongeek Shin , Shahram Izadi , David Kim , Sofien Bouaziz , Steven Benjamin Goldberg , Ivan Poupyrev , Shwetak N. Patel
摘要: Techniques of identifying gestures include detecting and classifying inner-wrist muscle motions at a user's wrist using micron-resolution radar sensors. For example, a user of an AR system may wear a band around their wrist. When the user makes a gesture to manipulate a virtual object in the AR system as seen in a head-mounted display (HMD), muscles and ligaments in the user's wrist make small movements on the order of 1-3 mm. The band contains a small radar device that has a transmitter and a number of receivers (e.g., three) of electromagnetic (EM) radiation on a chip (e.g., a Soli chip. This radiation reflects off the wrist muscles and ligaments and is received by the receivers on the chip in the band. The received reflected signal, or signal samples, are then sent to processing circuitry for classification to identify the wrist movement as a gesture.
-
-
-
-
-
-
-
-
-