公开(公告)号：US20240062008A1

公开(公告)日：2024-02-22

申请号：US17820437

申请日：2022-08-17

Applicant: Snap Inc.

Inventor： Arnab Ghosh ,  Jian Ren ,  Pavel Savchenkov ,  Sergey Tulyakov

IPC: G06F40/289 ,  G06T11/60 ,  H04L51/10 ,  G06F3/04842 ,  G06F16/583

CPC classification number: G06F40/289 ,  G06T11/60 ,  H04L51/10 ,  G06F3/04842 ,  G06F16/5846 ,  G06T2200/24

Abstract: A method of generating an image for use in a conversation taking place in a messaging application is disclosed. Conversation input text is received from a user of a portable device that includes a display. Model input text is generated from the conversation input text, which is processed with a text-to-image model to generate an image based on the model input text. The generated image is displayed on the portable device, and user input is received to transmit the image to a remote recipient.

公开(公告)号：US11836835B2

公开(公告)日：2023-12-05

申请号：US17364218

申请日：2021-06-30

Applicant: Snap Inc.

Inventor： Menglei Chai ,  Jian Ren ,  Aliaksandr Siarohin ,  Sergey Tulyakov ,  Oliver Woodford

IPC: G06T13/00 ,  G06N3/08 ,  G06T7/20 ,  G06T11/00 ,  G06N3/045

CPC classification number: G06T13/00 ,  G06N3/045 ,  G06N3/08 ,  G06T7/20 ,  G06T11/001 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.

公开(公告)号：US20220207329A1

公开(公告)日：2022-06-30

申请号：US17558327

申请日：2021-12-21

Applicant: Snap Inc.

Inventor： Jian Ren ,  Oliver Woodford ,  Sergey Tulyakov ,  Jiazhuo Wang ,  Qing Jin

IPC: G06N3/04 ,  G06N3/08

Abstract: Systems and methods herein describe an image compression system. The image compression system generates a first generative adversarial network (GAN), identifies a threshold, based on the threshold, generates a second GAN by pruning channels of the first GAN, trains the second GAN using similarity-based knowledge distillation from the first GAN, and stores the trained second GAN.

公开(公告)号：US20220172003A1

公开(公告)日：2022-06-02

申请号：US17547548

申请日：2021-12-10

Applicant: Snap Inc.

Inventor： Jianchao Yang ,  Ning Xu ,  Jian Ren

IPC: G06K9/62

Abstract: Disclosed herein are arrangements that facilitate the transfer of knowledge from models for a source data-processing domain to models for a target data-processing domain. A convolutional neural network space for a source domain is factored into a first classification space and a first reconstruction space. The first classification space stores class information and the first reconstruction space stores domain-specific information. A convolutional neural network space for a target domain is factored into a second classification space and a second reconstruction space. The second classification space stores class information and the second reconstruction space stores domain-specific information. Distribution of the first classification space and the second classification space is aligned.

公开(公告)号：US20220101104A1

公开(公告)日：2022-03-31

申请号：US17491226

申请日：2021-09-30

Applicant: Snap Inc.

Inventor： Menglei Chai ,  Kyle Olszewski ,  Jian Ren ,  Yu Tian ,  Sergey Tulyakov

IPC: G06N3/04 ,  G06K9/62 ,  G06N3/08 ,  G06T7/20

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for video synthesis. The program and method provide for accessing a primary generative adversarial network (GAN) comprising a pre-trained image generator, a motion generator comprising a plurality of neural networks, and a video discriminator; generating an updated GAN based on the primary GAN, by performing operations comprising identifying input data of the updated GAN, the input data comprising an initial latent code and a motion domain dataset, training the motion generator based on the input data, and adjusting weights of the plurality of neural networks of the primary GAN based on an output of the video discriminator; and generating a synthesized video based on the primary GAN and the input data.

公开(公告)号：US20220058880A1

公开(公告)日：2022-02-24

申请号：US17445549

申请日：2021-08-20

Applicant: Snap Inc.

Inventor： Artem Bondich ,  Menglei Chai ,  Oleksandr Pyshchenko ,  Jian Ren ,  Sergey Tulyakov

IPC: G06T19/00 ,  G06T19/20 ,  G06T17/00 ,  G06K9/62 ,  G06T7/246 ,  G06N3/04 ,  G06N3/08

Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.