-
公开(公告)号：US20230306675A1
公开(公告)日：2023-09-28
申请号：US17656778
申请日：2022-03-28
Applicant: Snap Inc.
Inventor： Zeng Huang , Jian Ren , Sergey Tulyakov , Menglei Chai , Kyle Olszewski , Huan Wang
CPC classification number: G06T15/06 , G06T7/97 , G06T2207/20081 , G06T2207/20084
Abstract: Methods and systems are disclosed for performing operations for generating a 3D model of a scene. The operations include: receiving a set of two-dimensional (2D) images representing a first view of a real-world environment; applying a machine learning model comprising a neural light field network to the set of 2D images to predict pixel values of a target image representing a second view of the real-world environment, the machine learning model being trained to map a ray origin and direction directly to a given pixel value; and generating a three-dimensional (3D) model of the real-world environment based on the set of 2D images and the predicted target image.
-
公开(公告)号：US20230079136A1
公开(公告)日：2023-03-16
申请号：US17987285
申请日：2022-11-15
Applicant: Snap Inc.
Inventor： Artem Bondich , Menglei Chai , Oleksandr Pyshchenko , Jian Ren , Sergey Tulyakov
Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.
-
公开(公告)号：US11521362B2
公开(公告)日：2022-12-06
申请号：US17445549
申请日：2021-08-20
Applicant: Snap Inc.
Inventor： Artem Bondich , Menglei Chai , Oleksandr Pyshchenko , Jian Ren , Sergey Tulyakov
Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.
-
公开(公告)号：US20220207786A1
公开(公告)日：2022-06-30
申请号：US17557834
申请日：2021-12-21
Applicant: Snap Inc.
Inventor： Jian Ren , Menglei Chai , Oliver Woodford , Kyle Olszewski , Sergey Tulyakov
Abstract: Systems and methods herein describe a motion retargeting system. The motion retargeting system accesses a plurality of two-dimensional images comprising a person performing a plurality of body poses, extracts a plurality of implicit volumetric representations from the plurality of body poses, generates a three-dimensional warping field, the three-dimensional warping field configured to warp the plurality of implicit volumetric representations from a canonical pose to a target pose, and based on the three-dimensional warping field, generates a two-dimensional image of an artificial person performing the target pose.
-
公开(公告)号：US11200459B1
公开(公告)日：2021-12-14
申请号：US16156882
申请日：2018-10-10
Applicant: Snap Inc.
Inventor： Jianchao Yang , Ning Xu , Jian Ren
Abstract: Disclosed herein are arrangements that facilitate the transfer of knowledge from models for a source data-processing domain to models for a target data-processing domain. A convolutional neural network space for a source domain is factored into a first classification space and a first reconstruction space. The first classification space stores class information and the first reconstruction space stores domain-specific information. A convolutional neural network space for a target domain is factored into a second classification space and a second reconstruction space. The second classification space stores class information and the second reconstruction space stores domain-specific information. Distribution of the first classification space and the second classification space is aligned.
-
Patent Agency Ranking
公开(公告)号：US20240394933A1
公开(公告)日：2024-11-28
申请号：US18596452
申请日：2024-03-05
Applicant: Snap Inc.
Inventor： Pavlo Chemerys , Colin Eles , Ju Hu , Qing Jin , Yanyu Li , Ergeta Muca , Jian Ren , Dhritiman Sagar , Aleksei Stoliar , Sergey Tulyakov , Huan Wang
IPC: G06T11/00
Abstract: Described is a system for improving machine learning models by accessing a first latent diffusion machine learning model, accessing a second latent diffusion machine learning model that was derived from the first latent diffusion machine learning model, the second latent diffusion machine learning model trained to perform a second number of denoising steps, generating noise data, processing the noise data via the first latent diffusion machine learning model to generate one or more first latent features, processing the noise data via the second latent diffusion machine learning model to generate one or more second latent features, and inputting the one or more first latent features and the one or more second latent features into a loss function. The system then modifies a parameter of the second latent diffusion machine learning model based on the output of the loss function.
-
公开(公告)号：US12124803B2
公开(公告)日：2024-10-22
申请号：US17820437
申请日：2022-08-17
Applicant: Snap Inc.
Inventor： Arnab Ghosh , Jian Ren , Pavel Savchenkov , Sergey Tulyakov
IPC: G06F40/289 , G06F3/04842 , G06F16/583 , G06T11/60 , H04L51/10
CPC classification number: G06F40/289 , G06F3/04842 , G06F16/5846 , G06T11/60 , H04L51/10 , G06T2200/24
Abstract: A method of generating an image for use in a conversation taking place in a messaging application is disclosed. Conversation input text is received from a user of a portable device that includes a display. Model input text is generated from the conversation input text, which is processed with a text-to-image model to generate an image based on the model input text. The generated image is displayed on the portable device, and user input is received to transmit the image to a remote recipient.
-
公开(公告)号：US12094073B2
公开(公告)日：2024-09-17
申请号：US17814391
申请日：2022-07-22
Applicant: Snap Inc.
Inventor： Menglei Chai , Sergey Tulyakov , Jian Ren , Hsin-Ying Lee , Kyle Olszewski , Zeng Huang , Zezhou Cheng
CPC classification number: G06T19/20 , G06T17/00 , G06T2219/2012 , G06T2219/2021
Abstract: Systems, computer readable media, and methods herein describe an editing system where a three-dimensional (3D) object can be edited by editing a 2D sketch or 2D RGB views of the 3D object. The editing system uses multi-modal (MM) variational auto-decoders (VADs)(MM-VADs) that are trained with a shared latent space that enables editing 3D objects by editing 2D sketches of the 3D objects. The system determines a latent code that corresponds to an edited or sketched 2D sketch. The latent code is then used to generate a 3D object using the MM-VADs with the latent code as input. The latent space is divided into a latent space for shapes and a latent space for colors. The MM-VADs are trained with variational auto-encoders (VAE) and a ground truth.
-
公开(公告)号：US11995781B2
公开(公告)日：2024-05-28
申请号：US17987285
申请日：2022-11-15
Applicant: Snap Inc.
Inventor： Artem Bondich , Menglei Chai , Oleksandr Pyshchenko , Jian Ren , Sergey Tulyakov
IPC: G06T19/00 , G06F18/213 , G06F18/214 , G06N3/045 , G06N3/08 , G06T7/246 , G06T17/00 , G06T19/20
CPC classification number: G06T19/006 , G06F18/213 , G06F18/214 , G06N3/045 , G06N3/08 , G06T7/251 , G06T17/00 , G06T19/20 , G06T2207/20081 , G06T2207/20084 , G06T2215/16 , G06T2219/024 , G06T2219/2024
Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.
-
公开(公告)号：US20240104789A1
公开(公告)日：2024-03-28
申请号：US17950945
申请日：2022-09-22
Applicant: Snap Inc.
Inventor： Arnab Ghosh , Jian Ren , Pavel Savchenkov , Sergey Tulyakov
IPC: G06T11/00 , G06F40/289 , G06F40/35 , G06V40/16
CPC classification number: G06T11/00 , G06F40/289 , G06F40/35 , G06V40/161
Abstract: A method of generating an image for use in a conversation taking place in a messaging application is disclosed. Conversation input text is received from a user of a portable device that includes a display. Model input text is generated from the conversation input text, which is processed with a text-to-image model to generate an image based on the model input text. The coordinates of a face in the image are determined, and the face of the user or another person is added to the image at the location. The final image is displayed on the portable device, and user input is received to transmit the image to a remote recipient.
-
-
-
-
-
-
-
-
-
Search Results
36
records
(took 0.018 seconds)
