-
公开(公告)号:US12125129B2
公开(公告)日:2024-10-22
申请号:US18136470
申请日:2023-04-19
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Aleksei Podkin , Aliaksandr Siarohin , Aleksei Stoliar , Sergey Tulyakov
CPC classification number: G06T13/00 , G06N3/045 , G06N3/08 , G06V40/171 , G06V40/174
Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.
-
12.
公开(公告)号:US12062144B2
公开(公告)日:2024-08-13
申请号:US17804500
申请日:2022-05-27
Applicant: Snap Inc.
Inventor: Konstantin Gudkov , Andrey Alejandrovich Gomez Zharkov , Vadim Velicodnii , Aleksei Zhuravlev , Sergey Demyanov
IPC: G06N20/00 , G06F3/01 , G06F3/04815 , G06F8/34 , G06N3/045 , G06N3/08 , G06T3/00 , G06T3/10 , G06T19/00
CPC classification number: G06T19/006 , G06F3/011 , G06F3/04815 , G06F8/34 , G06N3/045 , G06N3/08 , G06N20/00 , G06T3/10
Abstract: Methods and systems are disclosed for performing automatically creating AR experiences on a messaging platform. The methods and systems perform operations that include: receiving, via a graphical user interface (GUI), input that specifies a plurality of image transformation parameters; accessing a set of sample source images; modifying the set of sample source images based on the plurality of image transformation parameters to generate a set of sample target images; training a machine learning model to generate a given target image from a given source image by establishing a relationship between the set of sample source images and the set of sample target images; and automatically generating an augmented reality experience comprising the trained machine learning model.
-
公开(公告)号:US20240005617A1
公开(公告)日:2024-01-04
申请号:US17855179
申请日:2022-06-30
Applicant: Snap Inc.
Inventor: Vladislav Shakhrai , Sergey Demyanov , Mikhail Vasilkovskii , Aleksei Stoliar
CPC classification number: G06T19/20 , G06T11/001 , G06T17/20 , G06T7/40 , G06T2219/2016 , G06T2210/12 , G06T2210/22 , G06T2207/20081 , G06T2207/20084
Abstract: Methods and systems are disclosed for performing operations for generating a photorealistic rendering of an object. The operations include: accessing a set of albedo textures and a machine learning model associated with a real-world object, the set of albedo textures and a machine learning model having been trained based on a plurality of viewpoints of the real-world object; obtaining a three-dimensional (3D) mesh of the real-world object; receiving input that selects a new viewpoint that differs from the plurality of viewpoints of the real-world object; and generating a photorealistic rendering of the real-world object from the new viewpoint based on the 3D mesh of the real-world object, the set of albedo textures, and the machine learning model associated with the real-world object.
-
公开(公告)号:US20230252704A1
公开(公告)日:2023-08-10
申请号:US18136470
申请日:2023-04-19
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Aleksei Podkin , Aliaksandr Siarohin , Aleksei Stoliar , Sergey Tulyakov
CPC classification number: G06T13/00 , G06N3/08 , G06N3/045 , G06V40/174 , G06V40/171
Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.
-
公开(公告)号:US11645798B1
公开(公告)日:2023-05-09
申请号:US17303537
申请日:2021-06-01
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Aleksei Podkin , Aliaksandr Siarohin , Aleksei Stoliar , Sergey Tulyakov
CPC classification number: G06T13/00 , G06N3/0454 , G06N3/08 , G06V40/171 , G06V40/174
Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.
-
公开(公告)号:US20220207355A1
公开(公告)日:2022-06-30
申请号:US17318658
申请日:2021-05-12
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Konstantin Gudkov , Aleksei Stoliar , Roman Ushakov , Fedor Zhdanov
Abstract: Systems and methods herein describe an image manipulation system for generating modified images using a generative adversarial network. The image manipulation system accesses a pre-trained generative adversarial network (GAN), fine-tunes the pre-trained GAN by training a portion of existing neural network layers of the pre-trained GAN and newly added layers of the pre-trained GAN on a secondary image domain, adjusts the weights of the fine-tuned GAN using the weights of the pre-trained GAN, and stores the fine-tuned GAN. An image transformation system uses the generated modified images to train a subsequent neural network, which can access a face from a client device and transform it to a domain of images used for GAN fine-tuning.
-
公开(公告)号:US12148202B2
公开(公告)日:2024-11-19
申请号:US17841333
申请日:2022-06-15
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Konstantin Gudkov , Fedor Zhdanov , Andrei Zharkov
Abstract: An image manipulation system for generating modified images using a generative adversarial network (GAN) trains GANs using domain changes, aligns input images with generated images, classifies and associates target images based on a symmetry, and uses a modified discriminator structure. A method for domain changes includes generating, using a pre-trained GAN trained on a plurality of first target images, a plurality of images, and determining a feature for each of the plurality of images. The method further includes determining the feature for each of a plurality of second target images and matching, based on the feature, second target images of the plurality of second target images with the plurality of images. The method further includes training a discriminator of the pre-trained GAN with the second target images and the plurality of images.
-
公开(公告)号:US20230410438A1
公开(公告)日:2023-12-21
申请号:US17843573
申请日:2022-06-17
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Yunqing Hu , Istvan Marton , Daniil Ostashev , Aleksei Podkin
CPC classification number: G06T19/006 , G06T5/002 , G06V10/82 , G06V20/20 , G06V10/774 , G06T5/20 , G06T2207/20084 , G06T2207/20081 , G06T2207/30201
Abstract: Systems and embodiments herein describe an augmented reality (AR) object rendering system. The AR object rendering system receives an image, generates a set of noise parameters and a set of blur parameters for the image using a neural network trained on a paired dataset of images, identifies an AR object associated with the image, modifies the AR object using the set of noise parameters and the set of blur parameters, displays the modified augmented reality object within the image.
-
公开(公告)号:US11736717B2
公开(公告)日:2023-08-22
申请号:US17490277
申请日:2021-09-30
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Andrew Cheng-min Lin , Walton Lin , Aleksei Podkin , Aleksei Stoliar , Sergey Tulyakov
IPC: H04N19/54 , H04N19/137 , H04N19/149 , H04N19/172 , H04L65/70 , G06N3/045
CPC classification number: H04N19/54 , G06N3/045 , H04L65/70 , H04N19/137 , H04N19/149 , H04N19/172
Abstract: Systems and methods herein describe a video compression system. The described systems and methods accesses a sequence of image frames from a first computing device, the sequence of image frames comprising a first image frame and a second image frame, detects a first set of keypoints for the first image frame, transmits the first image frame and the first set of keypoints to a second computing device, detects a second set of keypoints for the second image frame, transmits the second set of keypoints to the second computing device, causes an animated image to be displayed on the second computing device.
-
公开(公告)号:US11657479B2
公开(公告)日:2023-05-23
申请号:US17445362
申请日:2021-08-18
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Aleksei Podkin , Aleksei Stoliar , Vadim Velicodnii , Fedor Zhdanov
CPC classification number: G06T5/001 , G06T5/10 , G06T2200/16 , G06T2207/10004 , G06T2207/20048 , G06T2207/20081 , G06T2207/30196 , H04L51/10
Abstract: A mobile device can implement a neural network-based domain transfer scheme to modify an image in a first domain appearance to a second domain appearance. The domain transfer scheme can be configured to detect an object in the image, apply an effect to the image, and blend the image using color space adjustments and blending schemes to generate a realistic result image. The domain transfer scheme can further be configured to efficiently execute on the constrained device by removing operational layers based on resources available on the mobile device.
-
-
-
-
-
-
-
-
-