-
公开(公告)号:US11354922B2
公开(公告)日:2022-06-07
申请号:US17138177
申请日:2020-12-30
Applicant: Snap Inc.
Inventor: Sergey Tulyakov , Roman Furko , Aleksei Stoliar
Abstract: A landmark detection system can more accurately detect landmarks in images using a detection scheme that penalizes for dispersion parameters, such as variance or scale. The landmark detection system can be trained using both labeled and unlabeled training data in a semi-supervised approach. The landmark detection system can further implement tracking of an object across multiple images using landmark data.
-
公开(公告)号:US20250148674A1
公开(公告)日:2025-05-08
申请号:US18982758
申请日:2024-12-16
Applicant: Snap Inc.
Inventor: Sergey Smetanin , Arnab Ghosh , Pavel Savchenkov , Jian Ren , Sergey Tulyakov , Ivan Babanin , Timur Zakirov , Roman Golobokov , Aleksandr Zakharov , Dor Ayalon , Nikita Demidov , Vladimir Gordienko , Daniel Moreno , Nikita Belosludtcev , Sofya Savinova
IPC: G06T11/60 , G06F3/0482
Abstract: Examples in the present disclosure relate to prompt-driven, conversation-specific image generation. A first user device of a first user of an interaction application transmits an image generation request. The interaction application enables the first user to interact with at least a second user. An image is automatically generated based on a prompt included in the image generation request, and the image is presented at the first user device. In response to receiving user input to select the image, the image is stored as a conversation-specific image in association with a first user profile of the first user and a second user profile of the second user. The conversation-specific image is presented at both the first user device of the first user and a second user device of the second user, together with conversation data associated with a conversation between the first user and the second user.
-
公开(公告)号:US20240394843A1
公开(公告)日:2024-11-28
申请号:US18434411
申请日:2024-02-06
Applicant: Snap Inc.
Inventor: Pavlo Chemerys , Colin Eles , Ju Hu , Qing Jin , Yanyu Li , Ergeta Muca , Jian Ren , Dhritiman Sagar , Aleksei Stoliar , Sergey Tulyakov , Huan Wang
Abstract: Described is a system for improving machine learning models by accessing a first latent diffusion machine learning model, the first latent diffusion machine learning model trained to perform a first number of denoising steps, accessing a second latent diffusion machine learning model that was derived from the first latent diffusion machine learning model, the second latent diffusion machine learning model trained to perform a second number of denoising steps, generating noise data, processing the noise data via the first latent diffusion machine learning model to generate one or more first images, processing the noise data via the second latent diffusion machine learning model to generate one or more second images, and modify a parameter of the second latent diffusion machine learning model based on a comparison of the one or more first images with the one or more second images.
-
公开(公告)号:US12154303B2
公开(公告)日:2024-11-26
申请号:US18238979
申请日:2023-08-28
Applicant: Snap Inc.
Inventor: Jian Ren , Menglei Chai , Sergey Tulyakov , Qing Jin
Abstract: System and methods for compressing image-to-image models. Generative Adversarial Networks (GANs) have achieved success in generating high-fidelity images. An image compression system and method adds a novel variant to class-dependent parameters (CLADE), referred to as CLADE-Avg, which recovers the image quality without introducing extra computational cost. An extra layer of average smoothing is performed between the parameter and normalization layers. Compared to CLADE, this image compression system and method smooths abrupt boundaries, and introduces more possible values for the scaling and shift. In addition, the kernel size for the average smoothing can be selected as a hyperparameter, such as a 3×3 kernel size. This method does not introduce extra multiplications but only addition, and thus does not introduce much computational overhead, as the division can be absorbed into the parameters after training.
-
公开(公告)号:US12125129B2
公开(公告)日:2024-10-22
申请号:US18136470
申请日:2023-04-19
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Aleksei Podkin , Aliaksandr Siarohin , Aleksei Stoliar , Sergey Tulyakov
CPC classification number: G06T13/00 , G06N3/045 , G06N3/08 , G06V40/171 , G06V40/174
Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.
-
16.
公开(公告)号:US20240282110A1
公开(公告)日:2024-08-22
申请号:US18653718
申请日:2024-05-02
Applicant: Snap Inc.
Inventor: Kavya Venkata Kota Kopparapu , Benjamin Dodson , Francesc Xavier Drudis Rius , Angus Kong , Richard Leider , Jien Ren , Sergey Tulyakov , Jiayao Yu
Abstract: Aspects of the present disclosure involve a system comprising a medium storing a program and method for machine-learning based selection of a representative video frame. The program and method provide for receiving a set of video frames; determining a first subset of frames by removing frames outside of an image quality threshold; determining a second subset by removing frames outside of an image stillness threshold; computing feature data for each frame in the second subset; providing, for each frame in the second subset, the feature data to a machine learning model (MLM), the MLM being configured to output a score for each frame in the second subset of frames based on the feature data, the MLM having been trained with a first set of images labeled based on aesthetics, and with a second set of images labeled based on image quality; and selecting a frame based on output scores.
-
17.
公开(公告)号:US12008811B2
公开(公告)日:2024-06-11
申请号:US17550852
申请日:2021-12-14
Applicant: Snap Inc.
Inventor: Kavya Venkata Kota Kopparapu , Benjamin Dodson , Francesc Xavier Drudis Rius , Angus Kong , Richard Leider , Jian Ren , Sergey Tulyakov , Jiayao Yu
Abstract: Aspects of the present disclosure involve a system comprising a medium storing a program and method for machine-learning based selection of a representative video frame. The program and method provide for receiving a set of video frames; determining a first subset of frames by removing frames outside of an image quality threshold; determining a second subset by removing frames outside of an image stillness threshold; computing feature data for each frame in the second subset; providing, for each frame in the second subset, the feature data to a machine learning model (MLM), the MLM being configured to output a score for each frame in the second subset of frames based on the feature data, the MLM having been trained with a first set of images labeled based on aesthetics, and with a second set of images labeled based on image quality; and selecting a frame based on output scores.
-
公开(公告)号:US11798261B2
公开(公告)日:2023-10-24
申请号:US17303871
申请日:2021-06-09
Applicant: Snap Inc.
Inventor: Chen Cao , Sergey Tulyakov , Zhenglin Geng
IPC: G06V40/16 , G06V10/82 , G06V10/764
CPC classification number: G06V10/764 , G06V10/82 , G06V40/161 , G06V40/168 , G06V40/172 , G06V40/175
Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for synthesizing a realistic image with a new expression of a face in an input image by receiving an input image comprising a face having a first expression; obtaining a target expression for the face; and extracting a texture of the face and a shape of the face. The program and method for generating, based on the extracted texture of the face, a target texture corresponding to the obtained target expression using a first machine learning technique; generating, based on the extracted shape of the face, a target shape corresponding to the obtained target expression using a second machine learning technique; and combining the generated target texture and generated target shape into an output image comprising the face having a second expression corresponding to the obtained target expression.
-
公开(公告)号:US11775158B2
公开(公告)日:2023-10-03
申请号:US17354520
申请日:2021-06-22
Applicant: Snap Inc.
Inventor: Theresa Barton , Yanping Chen , Jaewook Chung , Christopher Yale Crutchfield , Aymeric Damien , Sergei Kotcur , Igor Kudriashov , Sergey Tulyakov , Andrew Wan , Emre Yamangil
IPC: G06T7/11 , G06F3/04845 , G06F3/0482 , H04N5/265 , H04N5/262 , G06V20/40 , G06V40/16 , G06F3/04817
CPC classification number: G06F3/04845 , G06F3/0482 , G06T7/11 , G06V20/40 , G06V40/161 , H04N5/265 , H04N5/2628 , G06F3/04817 , G06T2207/10016 , G06T2207/20081 , G06T2207/20084 , G06T2207/20132 , G06T2207/30201
Abstract: A system of machine learning schemes can be configured to efficiently perform image processing tasks on a user device, such as a mobile phone. The system can selectively detect and transform individual regions within each frame of a live streaming video. The system can selectively partition and toggle image effects within the live streaming video.
-
公开(公告)号:US20230252704A1
公开(公告)日:2023-08-10
申请号:US18136470
申请日:2023-04-19
Applicant: Snap Inc.
Inventor: Sergey Demyanov , Aleksei Podkin , Aliaksandr Siarohin , Aleksei Stoliar , Sergey Tulyakov
CPC classification number: G06T13/00 , G06N3/08 , G06N3/045 , G06V40/174 , G06V40/171
Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.
-
-
-
-
-
-
-
-
-