公开(公告)号：US20240312122A1

公开(公告)日：2024-09-19

申请号：US18184459

申请日：2023-03-15

Applicant: Nvidia Corporation

Inventor： Nicolas Moenne-Loccoz ,  Zan Gojcic ,  Gavriel State ,  Zian Wang ,  Ignacio Llamas

IPC: G06T15/50 ,  G06T5/50 ,  G06V10/60 ,  G06V10/74 ,  G06V10/82

CPC classification number: G06T15/506 ,  G06T5/50 ,  G06V10/60 ,  G06V10/761 ,  G06V10/82 ,  G06T2207/20221

Abstract: Approaches presented herein provide for the generation of visual content, including different types of content representations from different sources, rendered to include consistent scene illumination for the various representations. A first render pass can produce a first image including only proxies of implicit representations (e.g., NeRF objects) under scene illumination. A second render pass can produce a second image that includes a representation of the explicit scene objects, as well as the proxies of the implicit representations, under the scene illumination, which produces secondary lighting effects. The first and second images are compared to determine irradiance ratio data for the various pixel locations. A third render pass can produce a third image that includes the implicit representations, which can have relighting performed according to the irradiance ratio data to include the secondary lighting effects. The implicit and explicit objects can then be composited to produce an image with consistent scene illumination.

公开(公告)号：US20240096017A1

公开(公告)日：2024-03-21

申请号：US17895793

申请日：2022-08-25

Applicant: Nvidia Corporation

Inventor： Jun Gao ,  Tianchang Shen ,  Zan Gojcic ,  Wenzheng Chen ,  Zian Wang ,  Daiqing Li ,  Or Litany ,  Sanja Fidler

IPC: G06T17/20

CPC classification number: G06T17/20 ,  G06T2207/10024 ,  G06T2207/20084

Abstract: Apparatuses, systems, and techniques are presented to generate digital content. In at least one embodiment, one or more neural networks are used to generate one or more textured three-dimensional meshes corresponding to one or more objects based, at least in part, one or more two-dimensional images of the one or more objects.

公开(公告)号：US20250095275A1

公开(公告)日：2025-03-20

申请号：US18630480

申请日：2024-04-09

Applicant: NVIDIA Corporation

Inventor： Zian Wang ,  Tianchang Shen ,  Jun Gao ,  Merlin Nimier-David ,  Thomas Müller-Höhne ,  Alexander Keller ,  Sanja Fidler ,  Zan Gojcic ,  Nicholas Mark Worth Sharp

IPC: G06T15/06 ,  G06V10/44

Abstract: In various examples, images (e.g., novel views) of an object may be rendered using an optimized number of samples of a 3D representation of the object. The optimized number of the samples may be determined based at least on casting rays into a scene that includes the 3D representation of the object and/or an acceleration data structure corresponding to the object. The acceleration data structure may include features corresponding to characteristics of the object, and the features may be indicative of the number of samples to be obtained from various portions of the 3D representation of the object to render the images. In some examples, the 3D representation may be a neural radiance field that includes, as a neural output, a spatially varying kernel size predicting the characteristics of the object, and the features of the acceleration data structure may be related to the spatially varying kernel size.

公开(公告)号：US20240296623A1

公开(公告)日：2024-09-05

申请号：US18169825

申请日：2023-02-15

Applicant: Nvidia Corporation

Inventor： Jiahui Huang ,  Francis Williams ,  Zan Gojcic ,  Matan Atzmon ,  Or Litany ,  Sanja Fidler

IPC: G06T17/20 ,  G06T15/08

CPC classification number: G06T17/20 ,  G06T15/08 ,  G06T2210/56

Abstract: Approaches presented herein provide for the reconstruction of implicit multi-dimensional shapes. In one embodiment, oriented point cloud data representative of an object can be obtained using a physical scanning process. The point cloud data can be provided as input to a trained density model that can infer density functions for various points. The points can be mapped to a voxel hierarchy, allowing density functions to be determined for those voxels at the various levels that are associated with at least one point of the input point cloud. Contribution weights can be determined for the various density functions for the sparse voxel hierarchy, and the weighted density functions combined to obtain a density field. The density field can be evaluated to generate a geometric mesh where points having a zero, or near-zero, value are determined to contribute to the surface of the object.

公开(公告)号：US20240362897A1

公开(公告)日：2024-10-31

申请号：US18634134

申请日：2024-04-12

Applicant: NVIDIA Corporation

Inventor： Tzofi Klinghoffer ,  Jonah Philion ,  Zan Gojcic ,  Sanja Fidler ,  Or Litany ,  Wenzheng Chen ,  Jose Manuel Alvarez Lopez

IPC: G06V10/774 ,  G06T7/55 ,  G06T15/20

CPC classification number: G06V10/774 ,  G06T7/55 ,  G06T15/205 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30181 ,  G06T2207/30252

Abstract: In various examples, systems and methods are disclosed relating to synthetic data generation using viewpoint augmentation for autonomous and semi-autonomous systems and applications. One or more circuits can identify a set of sequential images corresponding to a first viewpoint and generate a first transformed image corresponding to a second viewpoint using a first image of the set of sequential images as input to a machine-learning model. The one or more circuits can update the machine-learning model based at least on a loss determined according to the first transformed image and a second image of the set of sequential images.

公开(公告)号：US20240005604A1

公开(公告)日：2024-01-04

申请号：US18320716

申请日：2023-05-19

Applicant: Nvidia Corporation

Inventor： Karsten Julian Kreis ,  Xiaohui Zeng ,  Arash Vahdat ,  Francis Williams ,  Zan Gojcic ,  Or Litany ,  Sanja Fidler

IPC: G06T17/20 ,  G06V10/44

CPC classification number: G06T17/20 ,  G06V10/44

Abstract: Approaches presented herein provide for the unconditional generation of novel three dimensional (3D) object shape representations, such as point clouds or meshes. In at least one embodiment, a first denoising diffusion model (DDM) can be trained to synthesize a 1D shape latent from Gaussian noise, and a second DDM can be trained to generate a set of latent points conditioned on this 1D shape latent. The shape latent and set of latent points can be provided to a decoder to generate a 3D point cloud representative of a random object from among the object classes on which the models were trained. A surface reconstruction process may be used to generate a surface mesh from this generated point cloud. Such an approach can scale to complex and/or multimodal distributions, and can be highly flexible as it can be adapted to various tasks such as multimodal voxel- or text-guided synthesis.