-
公开(公告)号:US20220207786A1
公开(公告)日:2022-06-30
申请号:US17557834
申请日:2021-12-21
Applicant: Snap Inc.
Inventor: Jian Ren , Menglei Chai , Oliver Woodford , Kyle Olszewski , Sergey Tulyakov
Abstract: Systems and methods herein describe a motion retargeting system. The motion retargeting system accesses a plurality of two-dimensional images comprising a person performing a plurality of body poses, extracts a plurality of implicit volumetric representations from the plurality of body poses, generates a three-dimensional warping field, the three-dimensional warping field configured to warp the plurality of implicit volumetric representations from a canonical pose to a target pose, and based on the three-dimensional warping field, generates a two-dimensional image of an artificial person performing the target pose.
-
公开(公告)号:US20210311618A1
公开(公告)日:2021-10-07
申请号:US17354520
申请日:2021-06-22
Applicant: Snap Inc.
Inventor: Theresa Barton , Yanping Chen , Jaewook Chung , Christopher Yale Crutchfield , Aymeric Damien , Sergei Kotcur , Igor Kudriashov , Sergey Tulyakov , Andrew Wan , Emre Yamangil
IPC: G06F3/0484 , G06F3/0482 , H04N5/265 , G06K9/00 , G06T7/11 , H04N5/262
Abstract: A system of machine learning schemes can be configured to efficiently perform image processing tasks on a user device, such as a mobile phone. The system can selectively detect and transform individual regions within each frame of a live streaming video. The system can selectively partition and toggle image effects within the live streaming video.
-
公开(公告)号:US20210182624A1
公开(公告)日:2021-06-17
申请号:US17189563
申请日:2021-03-02
Applicant: Snap Inc.
Inventor: Sergey Tulyakov , Sergei Korolev , Aleksei Stoliar , Maksim Gusarov , Sergei Kotcur , Christopher Yale Crutchfield , Andrew Wan
Abstract: A compact generative neural network can be distilled from a teacher generative neural network using a training network. The compact network can be trained on the input data and output data of the teacher network. The training network train the student network using a discrimination layer and one or more types of losses, such as perception loss and adversarial loss.
-
公开(公告)号:US20240394933A1
公开(公告)日:2024-11-28
申请号:US18596452
申请日:2024-03-05
Applicant: Snap Inc.
Inventor: Pavlo Chemerys , Colin Eles , Ju Hu , Qing Jin , Yanyu Li , Ergeta Muca , Jian Ren , Dhritiman Sagar , Aleksei Stoliar , Sergey Tulyakov , Huan Wang
IPC: G06T11/00
Abstract: Described is a system for improving machine learning models by accessing a first latent diffusion machine learning model, accessing a second latent diffusion machine learning model that was derived from the first latent diffusion machine learning model, the second latent diffusion machine learning model trained to perform a second number of denoising steps, generating noise data, processing the noise data via the first latent diffusion machine learning model to generate one or more first latent features, processing the noise data via the second latent diffusion machine learning model to generate one or more second latent features, and inputting the one or more first latent features and the one or more second latent features into a loss function. The system then modifies a parameter of the second latent diffusion machine learning model based on the output of the loss function.
-
公开(公告)号:US12124803B2
公开(公告)日:2024-10-22
申请号:US17820437
申请日:2022-08-17
Applicant: Snap Inc.
Inventor: Arnab Ghosh , Jian Ren , Pavel Savchenkov , Sergey Tulyakov
IPC: G06F40/289 , G06F3/04842 , G06F16/583 , G06T11/60 , H04L51/10
CPC classification number: G06F40/289 , G06F3/04842 , G06F16/5846 , G06T11/60 , H04L51/10 , G06T2200/24
Abstract: A method of generating an image for use in a conversation taking place in a messaging application is disclosed. Conversation input text is received from a user of a portable device that includes a display. Model input text is generated from the conversation input text, which is processed with a text-to-image model to generate an image based on the model input text. The generated image is displayed on the portable device, and user input is received to transmit the image to a remote recipient.
-
公开(公告)号:US12094073B2
公开(公告)日:2024-09-17
申请号:US17814391
申请日:2022-07-22
Applicant: Snap Inc.
Inventor: Menglei Chai , Sergey Tulyakov , Jian Ren , Hsin-Ying Lee , Kyle Olszewski , Zeng Huang , Zezhou Cheng
CPC classification number: G06T19/20 , G06T17/00 , G06T2219/2012 , G06T2219/2021
Abstract: Systems, computer readable media, and methods herein describe an editing system where a three-dimensional (3D) object can be edited by editing a 2D sketch or 2D RGB views of the 3D object. The editing system uses multi-modal (MM) variational auto-decoders (VADs)(MM-VADs) that are trained with a shared latent space that enables editing 3D objects by editing 2D sketches of the 3D objects. The system determines a latent code that corresponds to an edited or sketched 2D sketch. The latent code is then used to generate a 3D object using the MM-VADs with the latent code as input. The latent space is divided into a latent space for shapes and a latent space for colors. The MM-VADs are trained with variational auto-encoders (VAE) and a ground truth.
-
公开(公告)号:US11995781B2
公开(公告)日:2024-05-28
申请号:US17987285
申请日:2022-11-15
Applicant: Snap Inc.
Inventor: Artem Bondich , Menglei Chai , Oleksandr Pyshchenko , Jian Ren , Sergey Tulyakov
IPC: G06T19/00 , G06F18/213 , G06F18/214 , G06N3/045 , G06N3/08 , G06T7/246 , G06T17/00 , G06T19/20
CPC classification number: G06T19/006 , G06F18/213 , G06F18/214 , G06N3/045 , G06N3/08 , G06T7/251 , G06T17/00 , G06T19/20 , G06T2207/20081 , G06T2207/20084 , G06T2215/16 , G06T2219/024 , G06T2219/2024
Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.
-
公开(公告)号:US20240104789A1
公开(公告)日:2024-03-28
申请号:US17950945
申请日:2022-09-22
Applicant: Snap Inc.
Inventor: Arnab Ghosh , Jian Ren , Pavel Savchenkov , Sergey Tulyakov
IPC: G06T11/00 , G06F40/289 , G06F40/35 , G06V40/16
CPC classification number: G06T11/00 , G06F40/289 , G06F40/35 , G06V40/161
Abstract: A method of generating an image for use in a conversation taking place in a messaging application is disclosed. Conversation input text is received from a user of a portable device that includes a display. Model input text is generated from the conversation input text, which is processed with a text-to-image model to generate an image based on the model input text. The coordinates of a face in the image are determined, and the face of the user or another person is added to the image at the location. The final image is displayed on the portable device, and user input is received to transmit the image to a remote recipient.
-
公开(公告)号:US20240062008A1
公开(公告)日:2024-02-22
申请号:US17820437
申请日:2022-08-17
Applicant: Snap Inc.
Inventor: Arnab Ghosh , Jian Ren , Pavel Savchenkov , Sergey Tulyakov
IPC: G06F40/289 , G06T11/60 , H04L51/10 , G06F3/04842 , G06F16/583
CPC classification number: G06F40/289 , G06T11/60 , H04L51/10 , G06F3/04842 , G06F16/5846 , G06T2200/24
Abstract: A method of generating an image for use in a conversation taking place in a messaging application is disclosed. Conversation input text is received from a user of a portable device that includes a display. Model input text is generated from the conversation input text, which is processed with a text-to-image model to generate an image based on the model input text. The generated image is displayed on the portable device, and user input is received to transmit the image to a remote recipient.
-
公开(公告)号:US11836835B2
公开(公告)日:2023-12-05
申请号:US17364218
申请日:2021-06-30
Applicant: Snap Inc.
Inventor: Menglei Chai , Jian Ren , Aliaksandr Siarohin , Sergey Tulyakov , Oliver Woodford
CPC classification number: G06T13/00 , G06N3/045 , G06N3/08 , G06T7/20 , G06T11/001 , G06T2207/20081 , G06T2207/20084
Abstract: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.
-
-
-
-
-
-
-
-
-