公开(公告)号：US20240020897A1

公开(公告)日：2024-01-18

申请号：US17862818

申请日：2022-07-12

Applicant: Nvidia Corporation

Inventor： Ting-Chun Wang ,  Ming-Yu Liu ,  Koki Nagano ,  Sameh Khamis ,  Jan Kautz

IPC: G06T11/60 ,  G06T5/00

CPC classification number: G06T11/60 ,  G06T5/006 ,  G06T2207/20084

Abstract: Apparatuses, systems, and techniques are presented to generate image data. In at least one embodiment, one or more neural networks are used to cause a lighting effect to be applied to one or more objects within one or more images based, at least in part, on synthetically generated images of the one or more objects.

公开(公告)号：US11775829B2

公开(公告)日：2023-10-03

申请号：US18079772

申请日：2022-12-12

Applicant: NVIDIA Corporation

Inventor： Tero Tapani Karras ,  Samuli Matias Laine ,  David Patrick Luebke ,  Jaakko T. Lehtinen ,  Miika Samuli Aittala ,  Timo Oskari Aila ,  Ming-Yu Liu ,  Arun Mohanray Mallya ,  Ting-Chun Wang

IPC: G06T9/00 ,  G06N3/08 ,  H04N19/20 ,  H04N7/15 ,  G06T7/73 ,  G06V40/16 ,  G06T5/00

CPC classification number: G06N3/08 ,  G06T5/003 ,  G06T7/73 ,  G06T9/002 ,  G06V40/168 ,  H04N7/157 ,  H04N19/20 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30201

Abstract: A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.

公开(公告)号：US20230147641A1

公开(公告)日：2023-05-11

申请号：US17522776

申请日：2021-11-09

Applicant: Nvidia Corporation

Inventor： Ting-Chun Wang ,  Tim Brooks ,  Ming-Yu Liu ,  Tero Karras ,  Jaakko Lehtinen

IPC: G06T13/00 ,  G06N3/04

CPC classification number: G06T13/00 ,  G06N3/0454

Abstract: Apparatuses, systems, and techniques are presented to reconstruct one or more images. In at least one embodiment, one or more neural networks are used to generate one or more images of one or more objects based, at least in part, on input indicating motion of the one or more objects.

公开(公告)号：US20230035306A1

公开(公告)日：2023-02-02

申请号：US17382027

申请日：2021-07-21

Applicant: Nvidia Corporation

Inventor： Ming-Yu Liu ,  Koki Nagano ,  Yeongho Seol ,  Jose Rafael Valle Gomes da Costa ,  Jaewoo Seo ,  Ting-Chun Wang ,  Arun Mallya ,  Sameh Khamis ,  Wei Ping ,  Rohan Badlani ,  Kevin Jonathan Shih ,  Bryan Catanzaro ,  Simon Yuen ,  Jan Kautz

IPC: G06T13/40 ,  H04N19/597 ,  G06N3/04 ,  G10L13/04 ,  G06T9/00 ,  G06T17/10 ,  G06T13/20

Abstract: Apparatuses, systems, and techniques are presented to generate media content. In at least one embodiment, a first neural network is used to generate first video information based, at least in part, upon voice information corresponding to one or more users, and a second neural network is used to generate second video information corresponding to the one or more users based, at least in part, upon the first video information and one or more images corresponding to the one or more users

公开(公告)号：US20210279841A1

公开(公告)日：2021-09-09

申请号：US16813589

申请日：2020-03-09

Applicant: NVIDIA Corporation

Inventor： Guilin Liu ,  Andrew Tao ,  Bryan Christopher Catanzaro ,  Ting-Chun Wang ,  Zhiding Yu ,  Shiqiu Liu ,  Fitsum Reda ,  Karan Sapra ,  Brandon Rowlett

IPC: G06T3/40 ,  G06T7/40 ,  G06K9/46 ,  G06N3/08

Abstract: Apparatuses, systems, and techniques for texture synthesis from small input textures in images using convolutional neural networks. In at least one embodiment, one or more convolutional layers are used in conjunction with one or more transposed convolution operations to generate a large textured output image from a small input textured image while preserving global features and texture, according to various novel techniques described herein.

公开(公告)号：US20210150187A1

公开(公告)日：2021-05-20

申请号：US17143516

申请日：2021-01-07

Applicant: NVIDIA Corporation

Inventor： Tero Tapani Karras ,  Samuli Matias Laine ,  David Patrick Luebke ,  Jaakko T. Lehtinen ,  Miika Samuli Aittala ,  Timo Oskari Aila ,  Ming-Yu Liu ,  Arun Mohanray Mallya ,  Ting-Chun Wang

IPC: G06K9/00 ,  G06N3/08 ,  G06T7/70

Abstract: A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.

公开(公告)号：US20200242771A1

公开(公告)日：2020-07-30

申请号：US16258322

申请日：2019-01-25

Applicant: Nvidia Corporation

Inventor： Taesung Park ,  Ming-Yu Liu ,  Ting-Chun Wang ,  Junyan Zhu

IPC: G06T7/11 ,  G06N3/04 ,  G06N3/08 ,  G06N20/10

Abstract: A user can create a basic semantic layout that includes two or more regions identified by the user, each region being associated with a semantic label indicating a type of object(s) to be rendered in that region. The semantic layout can be provided as input to an image synthesis network. The network can be a trained machine learning network, such as a generative adversarial network (GAN), that includes a conditional, spatially-adaptive normalization layer for propagating semantic information from the semantic layout to other layers of the network. The synthesis can involve both normalization and de-normalization, where each region of the layout can utilize different normalization parameter values. An image is inferred from the network, and rendered for display to the user. The user can change labels or regions in order to cause a new or updated image to be generated.

公开(公告)号：US20190244060A1

公开(公告)日：2019-08-08

申请号：US16265725

申请日：2019-02-01

Applicant: NVIDIA Corporation

Inventor： Aysegul Dundar ,  Ming-Yu Liu ,  Ting-Chun Wang ,  John Zedlewski ,  Jan Kautz

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04 ,  G06K9/32 ,  G06T3/00 ,  G06T7/10

CPC classification number: G06K9/6256 ,  G06K9/3233 ,  G06K9/6267 ,  G06N3/0454 ,  G06N3/08 ,  G06T3/0056 ,  G06T7/10

Abstract: A style transfer neural network may be used to generate stylized synthetic images, where real images provide the style (e.g., seasons, weather, lighting) for transfer to synthetic images. The stylized synthetic images may then be used to train a recognition neural network. In turn, the trained neural network may be used to predict semantic labels for the real images, providing recognition data for the real images. Finally, the real training dataset (real images and predicted recognition data) and the synthetic training dataset are used by the style transfer neural network to generate stylized synthetic images. The training of the neural network, prediction of recognition data for the real images, and stylizing of the synthetic images may be repeated for a number of iterations. The stylization operation more closely aligns a covariate of the synthetic images to the covariate of the real images, improving accuracy of the recognition neural network.

公开(公告)号：US20190147296A1

公开(公告)日：2019-05-16

申请号：US16188920

申请日：2018-11-13

Applicant: NVIDIA Corporation

Inventor： Ting-Chun Wang ,  Ming-Yu Liu ,  Bryan Christopher Catanzaro ,  Jan Kautz ,  Andrew J. Tao

IPC: G06K9/62 ,  G06K9/68 ,  G06K9/72

Abstract: A method, computer readable medium, and system are disclosed for creating an image utilizing a map representing different classes of specific pixels within a scene. One or more computing systems use the map to create a preliminary image. This preliminary image is then compared to an original image that was used to create the map. A determination is made whether the preliminary image matches the original image, and results of the determination are used to adjust the computing systems that created the preliminary image, which improves a performance of such computing systems. The adjusted computing systems are then used to create images based on different input maps representing various object classes of specific pixels within a scene.