公开(公告)号：US20240412458A1

公开(公告)日：2024-12-12

申请号：US18741680

申请日：2024-06-12

Applicant: Google LLC

Inventor： Varun Jampani ,  Chun-Han Yao ,  Amit Raj ,  Wei-Chih Hung ,  Ming-Hsuan Yang ,  Michael Rubinstein ,  Yuanzhen Li

IPC: G06T17/20 ,  G06T5/70 ,  G06T11/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for editing images based on decoder-based accumulative score sampling (DASS) losses.

公开(公告)号：US20240013497A1

公开(公告)日：2024-01-11

申请号：US18252118

申请日：2020-12-21

Applicant: Google LLC

Inventor： Deqing Sun ,  Varun Jampani ,  Gengshan Yang ,  Daniel Vlasic ,  Huiwen Chang ,  Forrester H. Cole ,  Ce Liu ,  William Tafel Freeman

IPC: G06T19/20 ,  G06T7/55 ,  G06T17/20 ,  G06T7/20 ,  G06T7/40

CPC classification number: G06T19/20 ,  G06T7/55 ,  G06T17/20 ,  G06T7/20 ,  G06T7/40 ,  G06T2207/30244 ,  G06T2207/10016 ,  G06T2207/20084 ,  G06T2219/2021 ,  G06T2207/20081

Abstract: A computing system and method can be used to render a 3D shape from one or more images. In particular, the present disclosure provides a general pipeline for learning articulated shape reconstruction from images (LASR). The pipeline can reconstruct rigid or nonrigid 3D shapes. In particular, the pipeline can automatically decompose non-rigidly deforming shapes into rigid motions near rigid-bones. This pipeline incorporates an analysis-by-synthesis strategy and forward-renders silhouette, optical flow, and color images which can be compared against the video observations to adjust the internal parameters of the model. By inverting a rendering pipeline and incorporating optical flow, the pipeline can recover a mesh of a 3D model from the one or more images input by a user.

公开(公告)号：US20240320912A1

公开(公告)日：2024-09-26

申请号：US18611236

申请日：2024-03-20

Applicant: Google LLC

Inventor： Yuanzhen Li ,  Amit Raj ,  Varun Jampani ,  Benjamin Joseph Mildenhall ,  Benjamin Michael Poole ,  Jonathan Tilton Barron ,  Kfir Aberman ,  Michael Niemeyer ,  Michael Rubinstein ,  Nataniel Ruiz Gutierrez ,  Shiran Elyahu Zada ,  Srinivas Kaza

IPC: G06T17/00 ,  H04N13/279 ,  H04N13/351

CPC classification number: G06T17/00 ,  H04N13/279 ,  H04N13/351

Abstract: A fractional training process can be performed training images to an instance of a machine-learned generative image model to obtain a partially trained instance of the model. A fractional optimization process can be performed with the partially trained instance to an instance of a machine-learned three-dimensional (3D) implicit representation model obtain a partially optimized instance of the model. Based on the plurality of training images, pseudo multi-view subject images can be generated with the partially optimized instance of the 3D implicit representation model and a fully trained instance of the generative image model; The partially trained instance of the model can be trained with a set of training data. The partially optimized instance of the machine-learned 3D implicit representation model can be trained with the machine-learned multi-view image model.

公开(公告)号：US20250166136A1

公开(公告)日：2025-05-22

申请号：US18957367

申请日：2024-11-22

Applicant: Google LLC

Inventor： Mark Jeffrey Matthews ,  Prafull Sharma ,  Dmitry Lagun ,  Xuhui Jia ,  Yuanzhen Li ,  Varun Jampani ,  William Tafel Freeman

IPC: G06T5/60 ,  G06T5/73 ,  G06T11/60

Abstract: Provided are systems and methods for controlling material attributes such as roughness, metallic, albedo, and transparency in real images. This method leverages the generative prior of text-to-image models known for their photorealistic capabilities, offering an alternative to traditional rendering pipelines. As one example, the technology can be used to alter the appearance of an object in an image, making it appear more metallic or changing its roughness to create a more matte or glossy finish. This can be particularly useful in various fields where the ability to manipulate the appearance of products in images can be a powerful tool.

公开(公告)号：US12260572B2

公开(公告)日：2025-03-25

申请号：US17907529

申请日：2021-08-05

Applicant: Google LLC

Inventor： Varun Jampani ,  Huiwen Chang ,  Kyle Sargent ,  Abhishek Kar ,  Richard Tucker ,  Dominik Kaeser ,  Brian L. Curless ,  David Salesin ,  William T. Freeman ,  Michael Krainin ,  Ce Liu

IPC: G06T7/00 ,  G06T5/60 ,  G06T5/77 ,  G06T7/50

Abstract: A method includes determining, based on an image having an initial viewpoint, a depth image, and determining a foreground visibility map including visibility values that are inversely proportional to a depth gradient of the depth image. The method also includes determining, based on the depth image, a background disocclusion mask indicating a likelihood that pixel of the image will be disoccluded by a viewpoint adjustment. The method additionally includes generating, based on the image, the depth image, and the background disocclusion mask, an inpainted image and an inpainted depth image. The method further includes generating, based on the depth image and the inpainted depth image, respectively, a first three-dimensional (3D) representation of the image and a second 3D representation of the inpainted image, and generating a modified image having an adjusted viewpoint by combining the first and second 3D representation based on the foreground visibility map.

公开(公告)号：US20240296596A1

公开(公告)日：2024-09-05

申请号：US18569844

申请日：2023-08-23

Applicant: Google LLC

Inventor： Kfir Aberman ,  Nataniel Ruiz Gutierrez ,  Michael Rubinstein ,  Yuanzhen Li ,  Yael Pritch Knaan ,  Varun Jampani

IPC: G06T11/00 ,  G06V10/764

CPC classification number: G06T11/00 ,  G06V10/764 ,  G06V2201/07

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text-to-image model so that the text-to-image model generates images that each depict a variable instance of an object class when the object class without the unique identifier is provided as a text input, and that generates images that each depict a same subject instance of the object class when the unique identifier is provided as the text input.

公开(公告)号：US20240249422A1

公开(公告)日：2024-07-25

申请号：US17907529

申请日：2021-08-05

Applicant: Google LLC

Inventor： Varun Jampani ,  Huiwen Chang ,  Kyle Sargent ,  Abhishek Kar ,  Richard Tucker ,  Dominik Kaeser ,  Brian L. Curless ,  David Salesin ,  William T. Freeman ,  Michael Krainin ,  Ce Liu

IPC: G06T7/50 ,  G06T5/60 ,  G06T5/77

CPC classification number: G06T7/50 ,  G06T5/60 ,  G06T5/77 ,  G06T2207/20081

Abstract: A method includes determining, based on an image having an initial viewpoint, a depth image, and determining a foreground visibility map including visibility values that are inversely proportional to a depth gradient of the depth image. The method also includes determining, based on the depth image, a background disocclusion mask indicating a likelihood that pixel of the image will be disoccluded by a viewpoint adjustment. The method additionally includes generating, based on the image, the depth image, and the background disocclusion mask, an inpainted image and an inpainted depth image. The method further includes generating, based on the depth image and the inpainted depth image, respectively, a first three-dimensional (3D) representation of the image and a second 3D representation of the inpainted image, and generating a modified image having an adjusted viewpoint by combining the first and second 3D representation based on the foreground visibility map.