公开(公告)号：US11736717B2

公开(公告)日：2023-08-22

申请号：US17490277

申请日：2021-09-30

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Andrew Cheng-min Lin ,  Walton Lin ,  Aleksei Podkin ,  Aleksei Stoliar ,  Sergey Tulyakov

IPC: H04N19/54 ,  H04N19/137 ,  H04N19/149 ,  H04N19/172 ,  H04L65/70 ,  G06N3/045

CPC classification number: H04N19/54 ,  G06N3/045 ,  H04L65/70 ,  H04N19/137 ,  H04N19/149 ,  H04N19/172

Abstract: Systems and methods herein describe a video compression system. The described systems and methods accesses a sequence of image frames from a first computing device, the sequence of image frames comprising a first image frame and a second image frame, detects a first set of keypoints for the first image frame, transmits the first image frame and the first set of keypoints to a second computing device, detects a second set of keypoints for the second image frame, transmits the second set of keypoints to the second computing device, causes an animated image to be displayed on the second computing device.

公开(公告)号：US11580400B1

公开(公告)日：2023-02-14

申请号：US16586635

申请日：2019-09-27

Applicant: Snap Inc.

Inventor： Enxu Yan ,  Sergey Tulyakov ,  Aleksei Podkin ,  Aleksei Stoliar

IPC: G06N3/08 ,  G06N3/082 ,  G06N3/04 ,  G06F17/16 ,  G06T7/10

Abstract: A neural network pruning system can sparsely prune neural network models using an optimizer based approach that is agnostic to the model architecture being pruned. The neural network pruning system can prune by operating on the parameter vector of the full model and the gradient vector of the loss function with respect to the model parameters. The neural network pruning system can iteratively update parameters based on the gradients, while zeroing out as many parameters as possible based a preconfigured penalty.

公开(公告)号：US20220207329A1

公开(公告)日：2022-06-30

申请号：US17558327

申请日：2021-12-21

Applicant: Snap Inc.

Inventor： Jian Ren ,  Oliver Woodford ,  Sergey Tulyakov ,  Jiazhuo Wang ,  Qing Jin

IPC: G06N3/04 ,  G06N3/08

Abstract: Systems and methods herein describe an image compression system. The image compression system generates a first generative adversarial network (GAN), identifies a threshold, based on the threshold, generates a second GAN by pruning channels of the first GAN, trains the second GAN using similarity-based knowledge distillation from the first GAN, and stores the trained second GAN.

公开(公告)号：US20220101104A1

公开(公告)日：2022-03-31

申请号：US17491226

申请日：2021-09-30

Applicant: Snap Inc.

Inventor： Menglei Chai ,  Kyle Olszewski ,  Jian Ren ,  Yu Tian ,  Sergey Tulyakov

IPC: G06N3/04 ,  G06K9/62 ,  G06N3/08 ,  G06T7/20

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for video synthesis. The program and method provide for accessing a primary generative adversarial network (GAN) comprising a pre-trained image generator, a motion generator comprising a plurality of neural networks, and a video discriminator; generating an updated GAN based on the primary GAN, by performing operations comprising identifying input data of the updated GAN, the input data comprising an initial latent code and a motion domain dataset, training the motion generator based on the input data, and adjusting weights of the plurality of neural networks of the primary GAN based on an output of the video discriminator; and generating a synthesized video based on the primary GAN and the input data.

公开(公告)号：US20220058880A1

公开(公告)日：2022-02-24

申请号：US17445549

申请日：2021-08-20

Applicant: Snap Inc.

Inventor： Artem Bondich ,  Menglei Chai ,  Oleksandr Pyshchenko ,  Jian Ren ,  Sergey Tulyakov

IPC: G06T19/00 ,  G06T19/20 ,  G06T17/00 ,  G06K9/62 ,  G06T7/246 ,  G06N3/04 ,  G06N3/08

Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.

公开(公告)号：US11068141B1

公开(公告)日：2021-07-20

申请号：US16265355

申请日：2019-02-01

Applicant: Snap Inc.

Inventor： Theresa Barton ,  Yanping Chen ,  Jaewook Chung ,  Christopher Yale Crutchfield ,  Aymeric Damien ,  Sergei Kotcur ,  Igor Kudriashov ,  Sergey Tulyakov ,  Andrew Wan ,  Emre Yamangil

IPC: G06F3/00 ,  G06F3/0484 ,  G06K9/00 ,  G06F3/0482 ,  G06T7/11 ,  H04N5/265 ,  H04N5/262 ,  G06F3/0481

Abstract: A system of machine learning schemes can be configured to efficiently perform image processing tasks on a user device, such as a mobile phone. The system can selectively detect and transform individual regions within each frame of a live streaming video. The system can selectively partition and toggle image effects within the live streaming video.

公开(公告)号：US11055514B1

公开(公告)日：2021-07-06

申请号：US16220859

申请日：2018-12-14

Applicant: Snap Inc.

Inventor： Chen Cao ,  Sergey Tulyakov ,  Zhenglin Geng

IPC: G06K9/00

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for synthesizing a realistic image with a new expression of a face in an input image by receiving an input image comprising a face having a first expression; obtaining a target expression for the face; and extracting a texture of the face and a shape of the face. The program and method for generating, based on the extracted texture of the face, a target texture corresponding to the obtained target expression using a first machine learning technique; generating, based on the extracted shape of the face, a target shape corresponding to the obtained target expression using a second machine learning technique; and combining the generated target texture and generated target shape into an output image comprising the face having a second expression corresponding to the obtained target expression.

公开(公告)号：US20210192198A1

公开(公告)日：2021-06-24

申请号：US17138177

申请日：2020-12-30

Applicant: Snap Inc.

Inventor： Sergey Tulyakov ,  Roman Furko ,  Aleksei Stoliar

IPC: G06K9/00 ,  G06K9/62 ,  G06T7/246 ,  G06T7/73 ,  G06T11/60

Abstract: A landmark detection system can more accurately detect landmarks in images using a detection scheme that penalizes for dispersion parameters, such as variance or scale. The landmark detection system can be trained using both labeled and unlabeled training data in a semi-supervised approach. The landmark detection system can further implement tracking of an object across multiple images using landmark data.