公开(公告)号：US20230154051A1

公开(公告)日：2023-05-18

申请号：US17919460

申请日：2020-04-17

Applicant: Google LLC

Inventor： Danhang Tang ,  Saurabh Singh ,  Cem Keskin ,  Phillip Andrew Chou ,  Christian Haene ,  Mingsong Dou ,  Sean Ryan Francesco Fanello ,  Jonathan Taylor ,  Andrea Tagliasacchi ,  Philip Lindsley Davidson ,  Yinda Zhang ,  Onur Gonen Guleryuz ,  Shahram Izadi ,  Sofien Bouaziz

IPC: G06T9/00

CPC classification number: G06T9/001 ,  G06T9/002

Abstract: Systems and methods are directed to encoding and/or decoding of the textures/geometry of a three-dimensional volumetric representation. An encoding computing system can obtain voxel blocks from a three-dimensional volumetric representation of an object. The encoding computing system can encode voxel blocks with a machine-learned voxel encoding model to obtain encoded voxel blocks. The encoding computing system can decode the encoded voxel blocks with a machine-learned voxel decoding model to obtain reconstructed voxel blocks. The encoding computing system can generate a reconstructed mesh representation of the object based at least in part on the one or more reconstructed voxel blocks. The encoding computing system can encode textures associated with the voxel blocks according to an encoding scheme and based at least in part on the reconstructed mesh representation of the object to obtain encoded textures.

公开(公告)号：US20240290025A1

公开(公告)日：2024-08-29

申请号：US18588948

申请日：2024-02-27

Applicant: GOOGLE LLC

Inventor： Yinda Zhang ,  Sean Ryan Francesco Fanello ,  Ziqian Bai ,  Feitong Tan ,  Zeng Huang ,  Kripasindhu Sarkar ,  Danhang Tang ,  Di Qiu ,  Abhimitra Meka ,  Ruofei Du ,  Mingsong Dou ,  Sergio Orts Escolano ,  Rohit Kumar Pandey ,  Thabo Beeler

IPC: G06T13/40 ,  G06T7/90 ,  G06T17/20 ,  G06V10/44

CPC classification number: G06T13/40 ,  G06T7/90 ,  G06T17/20 ,  G06V10/44 ,  G06T2207/10024 ,  G06T2207/20084

Abstract: A method comprises receiving a first sequence of images of a portion of a user, the first sequence of images being monocular images; generating an avatar based on the first sequence of images, the avatar being based on a model including a feature vector associated with a vertex; receiving a second sequence of images of the portion of the user; and based on the second sequence of images, modifying the avatar with a displacement of the vertex to represent a gesture of the avatar.

公开(公告)号：US20240212325A1

公开(公告)日：2024-06-27

申请号：US18596822

申请日：2024-03-06

Applicant: Google LLC

Inventor： Yinda Zhang ,  Feitong Tan ,  Danhang Tang ,  Mingsong Dou ,  Kaiwen Guo ,  Sean Ryan Francesco Fanello ,  Sofien Bouaziz ,  Cem Keskin ,  Ruofei Du ,  Rohit Kumar Pandey ,  Deqing Sun

IPC: G06V10/771 ,  G06T7/70 ,  G06T17/00 ,  G06V10/44 ,  G06V10/75

CPC classification number: G06V10/771 ,  G06T7/70 ,  G06T17/00 ,  G06V10/44 ,  G06V10/751 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for training models to predict dense correspondences across images such as human images. A model may be trained using synthetic training data created from one or more 3D computer models of a subject. In addition, one or more geodesic distances derived from the surfaces of one or more of the 3D models may be used to generate one or more loss values, which may in turn be used in modifying the model's parameters during training.

公开(公告)号：US11954899B2

公开(公告)日：2024-04-09

申请号：US18274371

申请日：2021-03-11

Applicant: Google LLC

Inventor： Yinda Zhang ,  Feitong Tan ,  Danhang Tang ,  Mingsong Dou ,  Kaiwen Guo ,  Sean Ryan Francesco Fanello ,  Sofien Bouaziz ,  Cem Keskin ,  Ruofei Du ,  Rohit Kumar Pandey ,  Deqing Sun

IPC: G06V10/771 ,  G06T7/70 ,  G06T17/00 ,  G06V10/44 ,  G06V10/75

CPC classification number: G06V10/771 ,  G06T7/70 ,  G06T17/00 ,  G06V10/44 ,  G06V10/751 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for training models to predict dense correspondences across images such as human images. A model may be trained using synthetic training data created from one or more 3D computer models of a subject. In addition, one or more geodesic distances derived from the surfaces of one or more of the 3D models may be used to generate one or more loss values, which may in turn be used in modifying the model's parameters during training.

公开(公告)号：US20240046618A1

公开(公告)日：2024-02-08

申请号：US18274371

申请日：2021-03-11

Applicant: Google LLC

Inventor： Yinda Zhang ,  Feitong Tan ,  Danhang Tang ,  Mingsong Dou ,  Kaiwen Guo ,  Sean Ryan Francesco Fanello ,  Sofien Bouaziz ,  Cem Keskin ,  Ruofei Du ,  Rohit Kumar Pandey ,  Deqing Sun

IPC: G06V10/771 ,  G06T17/00 ,  G06T7/70 ,  G06V10/44 ,  G06V10/75

CPC classification number: G06V10/771 ,  G06T17/00 ,  G06T7/70 ,  G06V10/44 ,  G06V10/751 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for training models to predict dense correspondences across images such as human images. A model may be trained using synthetic training data created from one or more 3D computer models of a subject. In addition, one or more geodesic distances derived from the surfaces of one or more of the 3D models may be used to generate one or more loss values, which may in turn be used in modifying the model's parameters during training.

公开(公告)号：US12066282B2

公开(公告)日：2024-08-20

申请号：US17413847

申请日：2020-11-11

Applicant: GOOGLE LLC

Inventor： Sean Ryan Francesco Fanello ,  Kaiwen Guo ,  Peter Christopher Lincoln ,  Philip Lindsley Davidson ,  Jessica L. Busch ,  Xueming Yu ,  Geoffrey Harvey ,  Sergio Orts Escolano ,  Rohit Kumar Pandey ,  Jason Dourgarian ,  Danhang Tang ,  Adarsh Prakash Murthy Kowdle ,  Emily B. Cooper ,  Mingsong Dou ,  Graham Fyffe ,  Christoph Rhemann ,  Jonathan James Taylor ,  Shahram Izadi ,  Paul Ernest Debevec

IPC: G01B11/25 ,  G01B11/245 ,  G06T15/50 ,  G06T17/20

CPC classification number: G01B11/2513 ,  G01B11/245 ,  G06T15/506 ,  G06T17/205

Abstract: A lighting stage includes a plurality of lights that project alternating spherical color gradient illumination patterns onto an object or human performer at a predetermined frequency. The lighting stage also includes a plurality of cameras that capture images of an object or human performer corresponding to the alternating spherical color gradient illumination patterns. The lighting stage also includes a plurality of depth sensors that capture depth maps of the object or human performer at the predetermined frequency. The lighting stage also includes (or is associated with) one or more processors that implement a machine learning algorithm to produce a three-dimensional (3D) model of the object or human performer. The 3D model includes relighting parameters used to relight the 3D model under different lighting conditions.

公开(公告)号：US20220065620A1

公开(公告)日：2022-03-03

申请号：US17413847

申请日：2020-11-11

Applicant: GOOGLE LLC

Inventor： Sean Ryan Francesco Fanello ,  Kaiwen Guo ,  Peter Christopher Lincoln ,  Philip Lindsley Davidson ,  Jessica L. Busch ,  Xueming Yu ,  Geoffrey Harvey ,  Sergio Orts Escolano ,  Rohit Kumar Pandey ,  Jason Dourgarian ,  Danhang Tang ,  Adarsh Prakash Murthy Kowdle ,  Emily B. Cooper ,  Mingsong Dou ,  Graham Fyffe ,  Christoph Rhemann ,  Jonathan James Taylor ,  Shahram Izadi ,  Paul Ernest Debevec

IPC: G01B11/25 ,  G06T15/50 ,  G01B11/245 ,  G06T17/20

Abstract: A lighting stage includes a plurality of lights that project alternating spherical color gradient illumination patterns onto an object or human performer at a predetermined frequency. The lighting stage also includes a plurality of cameras that capture images of an object or human performer corresponding to the alternating spherical color gradient illumination patterns. The lighting stage also includes a plurality of depth sensors that capture depth maps of the object or human performer at the predetermined frequency. The lighting stage also includes (or is associated with) one or more processors that implement a machine learning algorithm to produce a three-dimensional (3D) model of the object or human performer. The 3D model includes relighting parameters used to relight the 3D model under different lighting conditions.

公开(公告)号：US10937182B2

公开(公告)日：2021-03-02

申请号：US15994471

申请日：2018-05-31

Applicant: Google LLC

Inventor： Mingsong Dou ,  Sean Ryan Fanello ,  Adarsh Prakash Murthy Kowdle ,  Christoph Rhemann ,  Sameh Khamis ,  Philip L. Davidson ,  Shahram Izadi ,  Vladimir Tankovich

IPC: G06T7/593 ,  G06T7/73 ,  G06T15/08 ,  G06T17/20

Abstract: An electronic device estimates a pose of one or more subjects in an environment based on estimating a correspondence between a data volume containing a data mesh based on a current frame captured by a depth camera and a reference volume containing a plurality of fused prior data frames based on spectral embedding and performing bidirectional non-rigid matching between the reference volume and the current data frame to refine the correspondence so as to support location-based functionality. The electronic device predicts correspondences between the data volume and the reference volume based on spectral embedding. The correspondences provide constraints that accelerate the convergence between the data volume and the reference volume. By tracking changes between the current data mesh frame and the reference volume, the electronic device avoids tracking failures that can occur when relying solely on a previous data mesh frame.