公开(公告)号：US20230379491A1

公开(公告)日：2023-11-23

申请号：US18230511

申请日：2023-08-04

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Andrew Cheng-min Lin ,  Walton Lin ,  Aleksei Podkin ,  Aleksei Stoliar ,  Sergey Tulyakov

IPC: H04N19/54 ,  H04N19/137 ,  H04N19/149 ,  H04N19/172 ,  H04L65/70 ,  G06N3/045

CPC classification number: H04N19/54 ,  H04N19/137 ,  H04N19/149 ,  H04N19/172 ,  H04L65/70 ,  G06N3/045

Abstract: Systems and methods herein describe a video compression system. The described systems and methods accesses a sequence of image frames from a first computing device, the sequence of image frames comprising a first image frame and a second image frame, detects a first set of keypoints for the first image frame, transmits the first image frame and the first set of keypoints to a second computing device, detects a second set of keypoints for the second image frame, transmits the second set of keypoints to the second computing device, causes an animated image to be displayed on the second computing device.

公开(公告)号：US20230120964A1

公开(公告)日：2023-04-20

申请号：US17974400

申请日：2022-10-26

Applicant: Snap Inc.

Inventor： Olha Rykhliuk ,  Jonathan ` Solichin ,  Aleksei Stoliar

IPC: G06T11/60 ,  G06V10/46 ,  G06V10/26 ,  G06F18/24

Abstract: Systems and methods herein describe receiving an image via an image capture device, using a machine learning model, generating an image augmentation decision, accessing an augmented reality content item, associating the generated image augmentation decision with the augmented reality content item, modifying the received image using the augmented reality content item and the associated image augmentation decision, and causing presentation of the modified image on a graphical user interface of a computing device.

公开(公告)号：US11521339B2

公开(公告)日：2022-12-06

申请号：US16946413

申请日：2020-06-19

Applicant: Snap Inc.

Inventor： Olha Rykhliuk ,  Jonathan Solichin ,  Aleksei Stoliar

IPC: G06T11/60 ,  G06V10/46 ,  G06V10/26 ,  G06K9/62

Abstract: Systems and methods herein describe receiving an image via an image capture device, using a machine learning model, generating an image augmentation decision, accessing an augmented reality content item, associating the generated image augmentation decision with the augmented reality content item, modifying the received image using the augmented reality content item and the associated image augmentation decision, and causing presentation of the modified image on a graphical user interface of a computing device.

公开(公告)号：US20210182624A1

公开(公告)日：2021-06-17

申请号：US17189563

申请日：2021-03-02

Applicant: Snap Inc.

Inventor： Sergey Tulyakov ,  Sergei Korolev ,  Aleksei Stoliar ,  Maksim Gusarov ,  Sergei Kotcur ,  Christopher Yale Crutchfield ,  Andrew Wan

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: A compact generative neural network can be distilled from a teacher generative neural network using a training network. The compact network can be trained on the input data and output data of the teacher network. The training network train the student network using a discrimination layer and one or more types of losses, such as perception loss and adversarial loss.

公开(公告)号：US10567321B2

公开(公告)日：2020-02-18

申请号：US16237296

申请日：2018-12-31

Applicant: Snap Inc.

Inventor： Grygoriy Kozhemiak ,  Oleksandr Pyshchenko ,  Victor Shaburov ,  Trevor Stephenson ,  Aleksei Stoliar

IPC: G06F15/16 ,  H04L12/58 ,  H04L29/08

Abstract: Systems and methods are provided for receiving a first media content item associated with a first interactive object of an interactive message, receiving a second media content item associated with a second interactive object of the interactive message, generating a third media content item based on the first media content item and second media content item, wherein the third media content item comprises combined features of the first media content item and the second media content item, and causing display of the generated third media content item.

公开(公告)号：US20250111628A1

公开(公告)日：2025-04-03

申请号：US18375332

申请日：2023-09-29

Applicant: Snap Inc.

Inventor： Songfang Han ,  Sergei Korolev ,  Hsin-Ying Lee ,  Aleksei Stoliar

IPC: G06T19/20 ,  G06T7/90

Abstract: An artificial intelligence (AI) network or neural network is trained to generate three-dimensional (3D) models or shapes with color from two-dimensional (2D) input images and input text describing the 3D model with color. Example methods include converting a first three-dimensional (3D) model from a first representation to a second representation, the second representation including color information for the 3D model and inputting the second representation into an encoder to generate a third representation having a lower dimension than the second representation. The method further includes inputting the third representation into a decoder to generate a fourth representation having a same dimension as the second representation and generating a second 3D model from the fourth representation. The method further includes determining losses between the first 3D model and the second 3D model and updating weights of the encoder and the decoder based on the losses.

公开(公告)号：US20240394843A1

公开(公告)日：2024-11-28

申请号：US18434411

申请日：2024-02-06

Applicant: Snap Inc.

Inventor： Pavlo Chemerys ,  Colin Eles ,  Ju Hu ,  Qing Jin ,  Yanyu Li ,  Ergeta Muca ,  Jian Ren ,  Dhritiman Sagar ,  Aleksei Stoliar ,  Sergey Tulyakov ,  Huan Wang

IPC: G06T5/60 ,  G06T5/70

Abstract: Described is a system for improving machine learning models by accessing a first latent diffusion machine learning model, the first latent diffusion machine learning model trained to perform a first number of denoising steps, accessing a second latent diffusion machine learning model that was derived from the first latent diffusion machine learning model, the second latent diffusion machine learning model trained to perform a second number of denoising steps, generating noise data, processing the noise data via the first latent diffusion machine learning model to generate one or more first images, processing the noise data via the second latent diffusion machine learning model to generate one or more second images, and modify a parameter of the second latent diffusion machine learning model based on a comparison of the one or more first images with the one or more second images.

公开(公告)号：US20240355010A1

公开(公告)日：2024-10-24

申请号：US18529550

申请日：2023-12-05

Applicant: Snap Inc.

Inventor： Bohdan Ahafonov ,  Matthew Hallberg ,  Sergei Korolev ,  William Miles Miller ,  Daria Skrypnyk ,  Aleksei Stoliar

IPC: G06T11/00 ,  G06T7/11 ,  G06T11/40 ,  G06T11/60 ,  G06V20/20 ,  G10L15/08 ,  G10L15/22

CPC classification number: G06T11/001 ,  G06T7/11 ,  G06T11/40 ,  G06T11/60 ,  G06V20/20 ,  G10L15/08 ,  G10L15/22 ,  G06T2210/16 ,  G10L2015/088

Abstract: Methods and systems are disclosed for generating an extended reality (XR) try-on experience. The methods and systems store, in a multimodal memory, interaction data representing use of one or more interaction functions including data in different modalities. The methods and systems detect an object depicted in an image captured by an interaction client and generate, by a machine learning model, a prompt based on the object depicted in the image and the interaction data in the multimodal memory. The methods and systems generate an artificial texture based on the prompt and modify a texture of the object depicted in the image using the artificial texture that has been generated based on the prompt.

公开(公告)号：US12125129B2

公开(公告)日：2024-10-22

申请号：US18136470

申请日：2023-04-19

Applicant: Snap Inc.

Inventor： Sergey Demyanov ,  Aleksei Podkin ,  Aliaksandr Siarohin ,  Aleksei Stoliar ,  Sergey Tulyakov

IPC: G06T13/00 ,  G06N3/045 ,  G06N3/08 ,  G06V40/16

CPC classification number: G06T13/00 ,  G06N3/045 ,  G06N3/08 ,  G06V40/171 ,  G06V40/174

Abstract: Systems and methods are disclosed for generating, a source image sequence using an image sensor of the computing device, the source image sequence comprising a plurality of source images depicting a head and face, identifying driving image sequence data to modify face image feature data in the source image sequence, generating, using an image transformation neural network, a modified source image sequence comprising a plurality of modified source images depicting modified versions of the head and face, and storing the modified source image sequence on the computing device.

公开(公告)号：US20240005617A1

公开(公告)日：2024-01-04

申请号：US17855179

申请日：2022-06-30

Applicant: Snap Inc.

Inventor： Vladislav Shakhrai ,  Sergey Demyanov ,  Mikhail Vasilkovskii ,  Aleksei Stoliar

IPC: G06T19/20 ,  G06T11/00 ,  G06T17/20 ,  G06T7/40

CPC classification number: G06T19/20 ,  G06T11/001 ,  G06T17/20 ,  G06T7/40 ,  G06T2219/2016 ,  G06T2210/12 ,  G06T2210/22 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods and systems are disclosed for performing operations for generating a photorealistic rendering of an object. The operations include: accessing a set of albedo textures and a machine learning model associated with a real-world object, the set of albedo textures and a machine learning model having been trained based on a plurality of viewpoints of the real-world object; obtaining a three-dimensional (3D) mesh of the real-world object; receiving input that selects a new viewpoint that differs from the plurality of viewpoints of the real-world object; and generating a photorealistic rendering of the real-world object from the new viewpoint based on the 3D mesh of the real-world object, the set of albedo textures, and the machine learning model associated with the real-world object.