公开(公告)号：US20230316613A1

公开(公告)日：2023-10-05

申请号：US18206985

申请日：2023-06-07

Applicant: Snap Inc.

Inventor： Artem Bondich ,  Oleksandr Pyshchenko

IPC: G06T11/60 ,  G06T11/00 ,  G06T5/00 ,  G06T11/20

CPC classification number: G06T11/60 ,  G06T11/001 ,  G06T5/002 ,  G06T11/203 ,  G06T2207/30201 ,  G06T2200/24 ,  G06T2207/10016

Abstract: Systems and methods are provided for rendering hair. The systems and methods include receiving hair spline data comprising coordinates of a plurality of hair strands; selecting a first hair strand of the plurality of hair strands; retrieving coordinates of the first hair strand; identifying based on the respective coordinates of the plurality of hair strands a second hair strand that is adjacent to the first hair strand; storing a reference to the second hair strand in association with the coordinates of the first hair strand; and generating one or more additional hair strands between the first hair strand and the second hair strand based on the coordinates of the first hair strand and the reference to the second hair strand.

公开(公告)号：US10360708B2

公开(公告)日：2019-07-23

申请号：US15199472

申请日：2016-06-30

Applicant: Snap Inc

Inventor： Artem Bondich ,  Volodymyr Maltsev

IPC: G09G5/00 ,  G06T11/60 ,  G06K9/00

Abstract: Systems, devices, media, and methods are presented for generating ideograms from a set of images received in an image stream. The systems and methods detect at least a portion of a face within the image and identify a set of facial landmarks within the portion of the face. The systems and methods determine one or more characteristics representing the portion of the face, in response to detecting the portion of the face. Based on the one or more characteristics and the set of facial landmarks, the systems and methods generate a representation of a face. The systems and methods position one or more graphical elements proximate to the graphical model of the face and generate an ideogram from the graphical model and the one or more graphical elements.

公开(公告)号：US11995781B2

公开(公告)日：2024-05-28

申请号：US17987285

申请日：2022-11-15

Applicant: Snap Inc.

Inventor： Artem Bondich ,  Menglei Chai ,  Oleksandr Pyshchenko ,  Jian Ren ,  Sergey Tulyakov

IPC: G06T19/00 ,  G06F18/213 ,  G06F18/214 ,  G06N3/045 ,  G06N3/08 ,  G06T7/246 ,  G06T17/00 ,  G06T19/20

CPC classification number: G06T19/006 ,  G06F18/213 ,  G06F18/214 ,  G06N3/045 ,  G06N3/08 ,  G06T7/251 ,  G06T17/00 ,  G06T19/20 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2215/16 ,  G06T2219/024 ,  G06T2219/2024

Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.

公开(公告)号：US20220058880A1

公开(公告)日：2022-02-24

申请号：US17445549

申请日：2021-08-20

Applicant: Snap Inc.

Inventor： Artem Bondich ,  Menglei Chai ,  Oleksandr Pyshchenko ,  Jian Ren ,  Sergey Tulyakov

IPC: G06T19/00 ,  G06T19/20 ,  G06T17/00 ,  G06K9/62 ,  G06T7/246 ,  G06N3/04 ,  G06N3/08

Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.

公开(公告)号：US11030786B1

公开(公告)日：2021-06-08

申请号：US16532221

申请日：2019-08-05

Applicant: Snap Inc.

Inventor： Artem Bondich ,  Oleksandr Pyshchenko

IPC: G09G5/02 ,  G06T11/60 ,  G06T11/00 ,  G06T5/00 ,  G06T11/20

Abstract: Systems and methods are provided for rendering hair. The systems and methods include receiving hair spline data comprising coordinates of a plurality of hair strands; selecting a first hair strand of the plurality of hair strands; retrieving coordinates of the first hair strand; identifying based on the respective coordinates of the plurality of hair strands a second hair strand that is adjacent to the first hair strand; storing a reference to the second hair strand in association with the coordinates of the first hair strand; and generating one or more additional hair strands between the first hair strand and the second hair strand based on the coordinates of the first hair strand and the reference to the second hair strand.

公开(公告)号：US20240282066A1

公开(公告)日：2024-08-22

申请号：US18653609

申请日：2024-05-02

Applicant: Snap Inc.

Inventor： Artem Bondich ,  Menglei Chai ,  Olekssandr Pyshchenko ,  Jian Ren ,  Sergey Tulyakov

IPC: G06T19/00 ,  G06F18/214 ,  G06N3/045 ,  G06N3/08 ,  G06T7/246 ,  G06T17/00 ,  G06T19/20

CPC classification number: G06T19/006 ,  G06F18/214 ,  G06N3/045 ,  G06N3/08 ,  G06T7/251 ,  G06T17/00 ,  G06T19/20 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2215/16 ,  G06T2219/024 ,  G06T2219/2024

Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.

公开(公告)号：US20240070986A1

公开(公告)日：2024-02-29

申请号：US18502634

申请日：2023-11-06

Applicant: Snap Inc.

Inventor： Matan Zohar ,  Artem Bondich ,  Yanli Zhao ,  Brian Fulkerson

IPC: G06T17/20 ,  G06T7/246 ,  G06T7/70 ,  G06T19/00

CPC classification number: G06T17/205 ,  G06T7/251 ,  G06T7/70 ,  G06T19/006 ,  G06T2207/10016

Abstract: Methods and systems are disclosed for performing operations for deforming an external mesh. The operations comprise receiving a video that includes a depiction of a real-world object. The operations comprise generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video. The operations comprise obtaining an external mesh associated with an Augmented-Reality (AR) element. The operations comprise accessing a plurality of deformation attributes associated with the external mesh, each attribute corresponding to a different deformation model. The operations comprise separately deforming, based on respective deformation models, a first portion of the external mesh and a second portion of the external mesh. The operations comprise modifying the video to include a display of the AR element based on the separately deformed first and second portions of the external mesh.

公开(公告)号：US20230115531A1

公开(公告)日：2023-04-13

申请号：US17498394

申请日：2021-10-11

Applicant: Snap Inc.

Inventor： Matan Zohar ,  Artem Bondich ,  Yanli Zhao ,  Brian Fulkerson

IPC: G06T17/20 ,  G06T19/00 ,  G06T7/246 ,  G06T7/70

Abstract: Methods and systems are disclosed for performing operations for deforming an external mesh. The operations comprise receiving a video that includes a depiction of a real-world object. The operations comprise generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video. The operations comprise obtaining an external mesh associated with an Augmented-Reality (AR) element. The operations comprise accessing a plurality of deformation attributes associated with the external mesh, each attribute corresponding to a different deformation model. The operations comprise separately deforming, based on respective deformation models, a first portion of the external mesh and a second portion of the external mesh. The operations comprise modifying the video to include a display of the AR element based on the separately deformed first and second portions of the external mesh.

公开(公告)号：US10984569B2

公开(公告)日：2021-04-20

申请号：US16433725

申请日：2019-06-06

Applicant: Snap Inc.

Inventor： Artem Bondich ,  Volodymyr Maltsev

IPC: G06T11/60 ,  G06K9/00

Abstract: Systems, devices, media, and methods are presented for generating ideograms from a set of images received in an image stream. The systems and methods detect at least a portion of a face within the image and identify a set of facial landmarks within the portion of the face. The systems and methods determine one or more characteristics representing the portion of the face, in response to detecting the portion of the face. Based on the one or more characteristics and the set of facial landmarks, the systems and methods generate a representation of a face. The systems and methods position one or more graphical elements proximate to the graphical model of the face and generate an ideogram from the graphical model and the one or more graphical elements.

公开(公告)号：US12272015B2

公开(公告)日：2025-04-08

申请号：US18653609

申请日：2024-05-02

Applicant: Snap Inc.

Inventor： Artem Bondich ,  Menglei Chai ,  Oleksandr Pyshchenko ,  Jian Ren ,  Sergey Tulyakov

IPC: G06T19/00 ,  G06F18/214 ,  G06N3/045 ,  G06N3/08 ,  G06T7/246 ,  G06T17/00 ,  G06T19/20

Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.