公开(公告)号：US20240013497A1

公开(公告)日：2024-01-11

申请号：US18252118

申请日：2020-12-21

Applicant: Google LLC

Inventor： Deqing Sun ,  Varun Jampani ,  Gengshan Yang ,  Daniel Vlasic ,  Huiwen Chang ,  Forrester H. Cole ,  Ce Liu ,  William Tafel Freeman

IPC: G06T19/20 ,  G06T7/55 ,  G06T17/20 ,  G06T7/20 ,  G06T7/40

CPC classification number: G06T19/20 ,  G06T7/55 ,  G06T17/20 ,  G06T7/20 ,  G06T7/40 ,  G06T2207/30244 ,  G06T2207/10016 ,  G06T2207/20084 ,  G06T2219/2021 ,  G06T2207/20081

Abstract: A computing system and method can be used to render a 3D shape from one or more images. In particular, the present disclosure provides a general pipeline for learning articulated shape reconstruction from images (LASR). The pipeline can reconstruct rigid or nonrigid 3D shapes. In particular, the pipeline can automatically decompose non-rigidly deforming shapes into rigid motions near rigid-bones. This pipeline incorporates an analysis-by-synthesis strategy and forward-renders silhouette, optical flow, and color images which can be compared against the video observations to adjust the internal parameters of the model. By inverting a rendering pipeline and incorporating optical flow, the pipeline can recover a mesh of a 3D model from the one or more images input by a user.

公开(公告)号：US20230298269A1

公开(公告)日：2023-09-21

申请号：US18014307

申请日：2020-08-31

Applicant: Google LLC

Inventor： Kyle Adam Genova ,  Daniel Vlasic ,  Forrester H. Cole

IPC: G06T17/20 ,  G06T7/90

CPC classification number: G06T17/20 ,  G06T7/90

Abstract: Systems and methods of the present disclosure are directed to a method that can include obtaining a 3D mesh comprising polygons and texture/shading data. The method can include rasterizing the 3D mesh to obtain a 2D raster comprising pixels and coordinates respectively associated with a subset of pixels. The method can include determining an initial color value for the subset of pixels based on the coordinates of the pixel and the associated shading/texture data. The method can include constructing a splat at the coordinates of a respective pixel. The method can include determining an updated color value for a respective pixel based on a weighting of the subset of splats to generate a 2D rendering of the 3D mesh based on the coordinates of a pixel and a splat.

公开(公告)号：US12288295B2

公开(公告)日：2025-04-29

申请号：US18014307

申请日：2020-08-31

Applicant: Google LLC

Inventor： Kyle Adam Genova ,  Daniel Vlasic ,  Forrester H. Cole

IPC: G06T17/20 ,  G06T7/90

Abstract: Systems and methods of the present disclosure are directed to a method that can include obtaining a 3D mesh comprising polygons and texture/shading data. The method can include rasterizing the 3D mesh to obtain a 2D raster comprising pixels and coordinates respectively associated with a subset of pixels. The method can include determining an initial color value for the subset of pixels based on the coordinates of the pixel and the associated shading/texture data. The method can include constructing a splat at the coordinates of a respective pixel. The method can include determining an updated color value for a respective pixel based on a weighting of the subset of splats to generate a 2D rendering of the 3D mesh based on the coordinates of a pixel and a splat.

公开(公告)号：US20240394284A1

公开(公告)日：2024-11-28

申请号：US18386803

申请日：2023-11-03

Applicant: Google LLC

Inventor： Daniel Vlasic ,  Yiming Gu ,  Daniel Hernandez Diaz ,  Ilaï Deutel ,  Xi Xiong ,  Tianli Yu ,  Joseph Pagadora ,  Mingyang Ling ,  Jill Daley ,  Guolong Su

IPC: G06F16/332 ,  G06F40/40 ,  G06T7/11 ,  G06V30/414

Abstract: An aspect of the disclosed technology is a system and process that are able to answer a document query as text and also provide the location in an image where the answer text is detected. In one aspect of the disclosed technology, a machine learning model combines vision and language features for joint learning.