公开(公告)号：US20230386158A1

公开(公告)日：2023-11-30

申请号：US17814391

申请日：2022-07-22

Applicant: Snap Inc.

Inventor： Menglei Chai ,  Sergey Tulyakov ,  Jian Ren ,  Hsin-Ying Lee ,  Kyle Olszewski ,  Zeng Huang ,  Zezhou Cheng

IPC: G06T19/20 ,  G06T17/00

CPC classification number: G06T19/20 ,  G06T17/00 ,  G06T2219/2012 ,  G06T2219/2021

Abstract: Systems, computer readable media, and methods herein describe an editing system where a three-dimensional (3D) object can be edited by editing a 2D sketch or 2D RGB views of the 3D object. The editing system uses multi-modal (MM) variational auto-decoders (VADs)(MM-VADs) that are trained with a shared latent space that enables editing 3D objects by editing 2D sketches of the 3D objects. The system determines a latent code that corresponds to an edited or sketched 2D sketch. The latent code is then used to generate a 3D object using the MM-VADs with the latent code as input. The latent space is divided into a latent space for shapes and a latent space for colors. The MM-VADs are trained with variational auto-encoders (VAE) and a ground truth.

公开(公告)号：US12094073B2

公开(公告)日：2024-09-17

申请号：US17814391

申请日：2022-07-22

Applicant: Snap Inc.

Inventor： Menglei Chai ,  Sergey Tulyakov ,  Jian Ren ,  Hsin-Ying Lee ,  Kyle Olszewski ,  Zeng Huang ,  Zezhou Cheng

IPC: G06T19/20 ,  G06T9/00 ,  G06T11/00 ,  G06T17/00

CPC classification number: G06T19/20 ,  G06T17/00 ,  G06T2219/2012 ,  G06T2219/2021

Abstract: Systems, computer readable media, and methods herein describe an editing system where a three-dimensional (3D) object can be edited by editing a 2D sketch or 2D RGB views of the 3D object. The editing system uses multi-modal (MM) variational auto-decoders (VADs)(MM-VADs) that are trained with a shared latent space that enables editing 3D objects by editing 2D sketches of the 3D objects. The system determines a latent code that corresponds to an edited or sketched 2D sketch. The latent code is then used to generate a 3D object using the MM-VADs with the latent code as input. The latent space is divided into a latent space for shapes and a latent space for colors. The MM-VADs are trained with variational auto-encoders (VAE) and a ground truth.