-
公开(公告)号:US20210049468A1
公开(公告)日:2021-02-18
申请号:US17069449
申请日:2020-10-13
Applicant: NVIDIA Corporation
Inventor: Tero Tapani Karras , Samuli Matias Laine , David Patrick Luebke , Jaakko T. Lehtinen , Miika Samuli Aittala , Timo Oskari Aila , Ming-Yu Liu , Arun Mohanray Mallya , Ting-Chun Wang
Abstract: A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.
-
公开(公告)号:US20200302250A1
公开(公告)日:2020-09-24
申请号:US16825199
申请日:2020-03-20
Applicant: Nvidia Corporation
Inventor: Hang Chu , Daiqing Li , David Jesus Acuna Marrero , Amlan Kar , Maria Shugrina , Ming-Yu Liu , Antonio Torralba Barriuso , Sanja Fidler
Abstract: A generative model can be used for generation of spatial layouts and graphs. Such a model can progressively grow these layouts and graphs based on local statistics, where nodes can represent spatial control points of the layout, and edges can represent segments or paths between nodes, such as may correspond to road segments. A generative model can utilize an encoder-decoder architecture where the encoder is a recurrent neural network (RNN) that encodes local incoming paths into a node and the decoder is another RNN that generates outgoing nodes and edges connecting an existing node to the newly generated nodes. Generation is done iteratively, and can finish once all nodes are visited or another end condition is satisfied. Such a model can generate layouts by additionally conditioning on a set of attributes, giving control to a user in generating the layout.
-
公开(公告)号:US10467763B1
公开(公告)日:2019-11-05
申请号:US16537986
申请日:2019-08-12
Applicant: NVIDIA Corporation
Inventor: Deqing Sun , Xiaodong Yang , Ming-Yu Liu , Jan Kautz
Abstract: A method, computer readable medium, and system are disclosed for estimating optical flow between two images. A first pyramidal set of features is generated for a first image and a partial cost volume for a level of the first pyramidal set of features is computed, by a neural network, using features at the level of the first pyramidal set of features and warped features extracted from a second image, where the partial cost volume is computed across a limited range of pixels that is less than a full resolution of the first image, in pixels, at the level. The neural network processes the features and the partial cost volume to produce a refined optical flow estimate for the first image and the second image.
-
44.
公开(公告)号:US20190158884A1
公开(公告)日:2019-05-23
申请号:US16191174
申请日:2018-11-14
Applicant: NVIDIA Corporation
Inventor: Yi-Hsuan Tsai , Ming-Yu Liu , Deqing Sun , Ming-Hsuan Yang , Jan Kautz
IPC: H04N19/85
Abstract: A method, computer readable medium, and system are disclosed for identifying residual video data. This data describes data that is lost during a compression of original video data. For example, the original video data may be compressed and then decompressed, and this result may be compared to the original video data to determine the residual video data. This residual video data is transformed into a smaller format by means of encoding, binarizing, and compressing, and is sent to a destination. At the destination, the residual video data is transformed back into its original format and is used during the decompression of the compressed original video data to improve a quality of the decompressed original video data.
-
45.
公开(公告)号:US20190156154A1
公开(公告)日:2019-05-23
申请号:US16188641
申请日:2018-11-13
Applicant: NVIDIA Corporation
Inventor: Wei-Chih Tu , Ming-Yu Liu , Varun Jampani , Deqing Sun , Ming-Hsuan Yang , Jan Kautz
Abstract: Segmentation is the identification of separate objects within an image. An example is identification of a pedestrian passing in front of a car, where the pedestrian is a first object and the car is a second object. Superpixel segmentation is the identification of regions of pixels within an object that have similar properties An example is identification of pixel regions having a similar color, such as different articles of clothing worn by the pedestrian and different components of the car. A pixel affinity neural network (PAN) model is trained to generate pixel affinity maps for superpixel segmentation. The pixel affinity map defines the similarity of two points in space. In an embodiment, the pixel affinity map indicates a horizonal affinity and vertical affinity for each pixel in the image. The pixel affinity map is processed to identify the superpixels.
-
公开(公告)号:US20180288431A1
公开(公告)日:2018-10-04
申请号:US15939098
申请日:2018-03-28
Applicant: NVIDIA Corporation
Inventor: Ming-Yu Liu , Xiaodong Yang , Jan Kautz , Sergey Tulyakov
IPC: H04N19/513 , G06K9/00 , G06N3/08 , G06T13/40
CPC classification number: H04N19/521 , G06K9/00201 , G06K9/00281 , G06N3/0445 , G06N3/0454 , G06N3/0472 , G06N3/08 , G06T13/40 , G06T2207/20081 , G06T2207/30196
Abstract: A method, computer readable medium, and system are disclosed for action video generation. The method includes the steps of generating, by a recurrent neural network, a sequence of motion vectors from a first set of random variables and receiving, by a generator neural network, the sequence of motion vectors and a content vector sample. The sequence of motion vectors and the content vector sample are sampled by the generator neural network to produce a video clip.
-
公开(公告)号:US20180247201A1
公开(公告)日:2018-08-30
申请号:US15907098
申请日:2018-02-27
Applicant: NVIDIA Corporation
Inventor: Ming-Yu Liu , Thomas Michael Breuel , Jan Kautz
CPC classification number: G06N3/088 , G06N3/0454 , G06N3/0472 , G06N3/063 , G06N3/084 , G06T1/00 , G06T3/4046
Abstract: A method, computer readable medium, and system are disclosed for training a neural network. The method includes the steps of encoding, by a first neural network, a first image represented in a first domain to convert the first image to a shared latent space, producing a first latent code and encoding, by a second neural network, a second image represented in a second domain to convert the second image to a shared latent space, producing a second latent code. The method also includes the step of generating, by a third neural network, a first translated image in the second domain based on the first latent code, wherein the first translated image is correlated with the first image and weight values of the third neural network are computed based on the first latent code and the second latent code.
-
公开(公告)号:US20250166237A1
公开(公告)日:2025-05-22
申请号:US18518430
申请日:2023-11-22
Applicant: NVIDIA Corporation
Inventor: Yu Zeng , Yogesh Balaji , Ting-Chun Wang , Xun Huang , Ming-Yu Liu
Abstract: Apparatuses, processors, computing systems, devices, non-transitory computer medium, and/or methods for using neural networks for generating multiple related images. In at least one embodiment, a processor includes circuitry to use one or more neural networks to generate several images, where each image includes a same object (e.g., same subject) and different backgrounds. For example, a processor including one or more circuits to use one or more neural networks to generate one or more objects (e.g., an animal, a vehicle, a person) within two or more different images (e.g., different backgrounds such as weather, season, environment) based, at least in part, on one or more indications (e.g., text prompts) by one or more users indicating content of at least one of the two or more different images (e.g., objects and/or backgrounds for each image in text such as adjectives and nouns) other than the one or more objects.
-
公开(公告)号:US20240338871A1
公开(公告)日:2024-10-10
申请号:US18746911
申请日:2024-06-18
Applicant: NVIDIA Corporation
Inventor: Donghoom LEE , Sifei Liu , Jinwei Gu , Ming-Yu Liu , Jan Kautz
CPC classification number: G06T11/60 , G06F18/217 , G06F18/24 , G06T3/02 , G06T7/30 , G06V30/274 , G06T7/70 , G06T2207/20081 , G06T2207/20084 , G06T2210/12
Abstract: One embodiment of a method includes applying a first generator model to a semantic representation of an image to generate an affine transformation, where the affine transformation represents a bounding box associated with at least one region within the image. The method further includes applying a second generator model to the affine transformation and the semantic representation to generate a shape of an object. The method further includes inserting the object into the image based on the bounding box and the shape.
-
50.
公开(公告)号:US20240296627A1
公开(公告)日:2024-09-05
申请号:US18662020
申请日:2024-05-13
Applicant: NVIDIA Corporation
Inventor: Tianchang Shen , Jun Gao , Kangxue Yin , Ming-Yu Liu , Sanja Fidler
CPC classification number: G06T17/20 , G06T7/50 , G06T2207/10028 , G06T2207/20081 , G06T2207/20084
Abstract: In various examples, a deep three-dimensional (3D) conditional generative model is implemented that can synthesize high resolution 3D shapes using simple guides—such as coarse voxels, point clouds, etc.—by marrying implicit and explicit 3D representations into a hybrid 3D representation. The present approach may directly optimize for the reconstructed surface, allowing for the synthesis of finer geometric details with fewer artifacts. The systems and methods described herein may use a deformable tetrahedral grid that encodes a discretized signed distance function (SDF) and a differentiable marching tetrahedral layer that converts the implicit SDF representation to an explicit surface mesh representation. This combination allows joint optimization of the surface geometry and topology as well as generation of the hierarchy of subdivisions using reconstruction and adversarial losses defined explicitly on the surface mesh.
-
-
-
-
-
-
-
-
-