公开(公告)号：US11948078B2

公开(公告)日：2024-04-02

申请号：US17000048

申请日：2020-08-21

Applicant: Nvidia Corporation

Inventor： Arash Vahdat ,  Tanmay Gupta ,  Xiaodong Yang ,  Jan Kautz

IPC: G06N3/08 ,  G06F18/214 ,  G06F18/22 ,  G06V10/74 ,  G06V10/82 ,  G06V30/19 ,  G06V30/262

CPC classification number: G06N3/08 ,  G06F18/2148 ,  G06F18/22 ,  G06V10/761 ,  G06V10/82 ,  G06V30/1916 ,  G06V30/19173 ,  G06V30/274

Abstract: The disclosure provides a framework or system for learning visual representation using a large set of image/text pairs. The disclosure provides, for example, a method of visual representation learning, a joint representation learning system, and an artificial intelligence (AI) system that employs one or more of the trained models from the method or system. The AI system can be used, for example, in autonomous or semi-autonomous vehicles. In one example, the method of visual representation learning includes: (1) receiving a set of image embeddings from an image representation model and a set of text embeddings from a text representation model, and (2) training, employing mutual information, a critic function by learning relationships between the set of image embeddings and the set of text embeddings.

公开(公告)号：US20230351807A1

公开(公告)日：2023-11-02

申请号：US17661706

申请日：2022-05-02

Applicant: NVIDIA Corporation

Inventor： Yuzhuo Ren ,  Weili Nie ,  Arash Vahdat ,  Animashree Anandkumar ,  Nishant Puri ,  Niranjan Avadhanam

IPC: G06V40/16 ,  G06V10/82 ,  G06V10/774 ,  G06V10/62

CPC classification number: G06V40/176 ,  G06V10/82 ,  G06V10/774 ,  G06V10/62 ,  G06V40/164

Abstract: A machine learning model (MLM) may be trained and evaluated. Attribute-based performance metrics may be analyzed to identify attributes for which the MLM is performing below a threshold when each are present in a sample. A generative neural network (GNN) may be used to generate samples including compositions of the attributes, and the samples may be used to augment the data used to train the MLM. This may be repeated until one or more criteria are satisfied. In various examples, a temporal sequence of data items, such as frames of a video, may be generated which may form samples of the data set. Sets of attribute values may be determined based on one or more temporal scenarios to be represented in the data set, and one or more GNNs may be used to generate the sequence to depict information corresponding to the attribute values.

公开(公告)号：US20230095092A1

公开(公告)日：2023-03-30

申请号：US17957143

申请日：2022-09-30

Applicant: Nvidia Corporation

Inventor： Zhisheng Xiao ,  Karsten Kreis ,  Arash Vahdat

IPC: G06T5/00

Abstract: Apparatuses, systems, and techniques are presented to train and utilize one or more neural networks. A denoising diffusion generative adversarial network (denoising diffusion GAN) reduces a number of denoising steps during a reverse process. The denoising diffusion GAN does not assume a Gaussian distribution for large steps of the denoising process and applies a multi-model model to permit denoising with fewer steps. Systems and methods further minimize a divergence between a diffused real data distribution and a diffused generator distribution over several timesteps. Accordingly, various embodiments may enable faster sample generation, in which the samples are generated from noise using the denoising diffusion GAN.

公开(公告)号：US20240412491A1

公开(公告)日：2024-12-12

申请号：US18207953

申请日：2023-06-09

Applicant: NVIDIA Corporation

Inventor： Shagan Sah ,  Nishant Puri ,  Yuzhuo Ren ,  Rajath Bellipady Shetty ,  Weili Nie ,  Arash Vahdat ,  Animashree Anandkumar

IPC: G06V10/776 ,  G06N3/094 ,  G06T11/00 ,  G06V10/75 ,  G06V10/774 ,  G06V10/82 ,  G06V40/16

Abstract: Apparatuses, system, and techniques use one or more first neural networks to generate one or more synthetic data to train one or more second neural networks based, at least in part, on one or more performance metrics of one or more second neural networks.

公开(公告)号：US20240253217A1

公开(公告)日：2024-08-01

申请号：US18538248

申请日：2023-12-13

Applicant: NVIDIA Corporation

Inventor： Arash Vahdat ,  Hongxu Yin ,  Jan Kautz ,  Jiaming Song ,  Ming-Yu Liu ,  Morteza Mardani ,  Qinsheng Zhang

IPC: B25J9/16

CPC classification number: B25J9/163 ,  B25J9/1664 ,  B25J9/1697

Abstract: Apparatuses, systems, and techniques to calculate a combined loss value based on applying one or more loss functions to the plurality of samples generated by a diffusion model to update the samples to determine a synthesized motions of one or more objects.

公开(公告)号：US20240005604A1

公开(公告)日：2024-01-04

申请号：US18320716

申请日：2023-05-19

Applicant: Nvidia Corporation

Inventor： Karsten Julian Kreis ,  Xiaohui Zeng ,  Arash Vahdat ,  Francis Williams ,  Zan Gojcic ,  Or Litany ,  Sanja Fidler

IPC: G06T17/20 ,  G06V10/44

CPC classification number: G06T17/20 ,  G06V10/44

Abstract: Approaches presented herein provide for the unconditional generation of novel three dimensional (3D) object shape representations, such as point clouds or meshes. In at least one embodiment, a first denoising diffusion model (DDM) can be trained to synthesize a 1D shape latent from Gaussian noise, and a second DDM can be trained to generate a set of latent points conditioned on this 1D shape latent. The shape latent and set of latent points can be provided to a decoder to generate a 3D point cloud representative of a random object from among the object classes on which the models were trained. A surface reconstruction process may be used to generate a surface mesh from this generated point cloud. Such an approach can scale to complex and/or multimodal distributions, and can be highly flexible as it can be adapted to various tasks such as multimodal voxel- or text-guided synthesis.

公开(公告)号：US20230377099A1

公开(公告)日：2023-11-23

申请号：US18319986

申请日：2023-05-18

Applicant: Nvidia Corporation

Inventor： Karsten Julian Kreis ,  Tim Dockhorn ,  Arash Vahdat

IPC: G06T5/00 ,  G06T7/64 ,  G06T11/00 ,  G06N3/045 ,  G06N3/08

CPC classification number: G06T5/002 ,  G06T7/64 ,  G06T11/00 ,  G06N3/045 ,  G06N3/08 ,  G06T2207/20084 ,  G06T2207/20081 ,  G06T2207/30241 ,  G06T2200/28

Abstract: Approaches presented herein provide for the generation of synthesized data from input noise using a denoising diffusion network. A higher order differential equation solver can be used for the denoising process, with one or more higher-order terms being distilled into one or more separate efficient neural networks. A separate, efficient neural network can be called together with a primary denoising model at inference time without significant loss in sampling efficiency. The separate neural network can provide information about the curvature (or other higher-order term) of the differential equation, representing a denoising trajectory, that can be used by the primary diffusion network to denoise the image using fewer denoising iterations.

公开(公告)号：US20230015253A1

公开(公告)日：2023-01-19

申请号：US17505384

申请日：2021-10-19

Applicant: Nvidia Corporation

Inventor： Weili Nie ,  Arash Vahdat ,  Anima Anandkumar

IPC: G06N3/08

Abstract: Apparatuses, systems, and techniques are presented to generate one or more images comprising one or more objects based, at least in part, on one or more dynamically configurable attributes of the one or objects. In at least one embodiment, one or more images comprising one or more objects can be generated based, at least in part, on one or more dynamically configurable attributes of the one or objects.

公开(公告)号：US20220318557A1

公开(公告)日：2022-10-06

申请号：US17224041

申请日：2021-04-06

Applicant: NVIDIA Corporation

Inventor： Sina Mohseni ,  Arash Vahdat ,  Jay Yadawa

IPC: G06K9/00 ,  G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: Apparatuses, systems, and techniques to identify out-of-distribution input data in one or more neural networks. In at least one embodiment, a technique includes training one or more neural networks to infer a plurality of characteristics about input information based, at least in part, on the one or more neural networks being independently trained to infer each of the plurality of characteristics about the input information.

公开(公告)号：US20220108213A1

公开(公告)日：2022-04-07

申请号：US17317698

申请日：2021-05-11

Applicant: NVIDIA Corporation

Inventor： Tianshi Cao ,  Alex Bie ,  Karsten Julian Kreis ,  Sanja Fidler ,  Arash Vahdat

IPC: G06N20/00 ,  G06K9/62 ,  G06N3/08 ,  G06F21/62

Abstract: Apparatuses, systems, and techniques to train a generative model based at least in part on a private dataset. In at least one embodiment, the generative model is trained based at least in part on a differentially private Sinkhorn algorithm, for example, using backpropagation with gradient descent to determine a gradient of a set of parameters of the generative models and modifying the set of parameters based at least in part on the gradient.