公开(公告)号：US20240104954A1

公开(公告)日：2024-03-28

申请号：US18534258

申请日：2023-12-08

Applicant: Snap Inc.

Inventor： Roman Golobokov ,  Alexandr Marinenko ,  Aleksandr Mashrabov ,  Aleksei Bromot ,  Grigoriy Tkachenko

IPC: G06V40/16 ,  G06T19/00

CPC classification number: G06V40/168 ,  G06T19/006 ,  G06V40/174 ,  G06T2200/24

Abstract: The subject technology captures first image data by a computing device, the first image data comprising a target face of a target actor and facial expressions of the target actor, the facial expressions including lip movements. The subject technology generates, based at least in part on frames of a source media content, sets of source pose parameters. The subject technology receives a selection of a particular facial expression from a set of facial expressions. The subject technology generates, based at least in part on sets of source pose parameters and the selection of the particular facial expression, an output media content. The subject technology provides augmented reality content based at least in part on the output media content for display on the computing device.

公开(公告)号：US20230351998A1

公开(公告)日：2023-11-02

申请号：US18218028

申请日：2023-07-04

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Maxim Lukin ,  Aleksandr Mashrabov

IPC: G10L13/00 ,  G10L13/08 ,  G06T13/40 ,  G06V40/16 ,  G06V10/764 ,  G06V10/82

CPC classification number: G10L13/00 ,  G10L13/08 ,  G06T13/40 ,  G06V40/171 ,  G06V10/764 ,  G06V10/82

Abstract: Systems and methods for text and audio-based real-time face reenactment are provided. An example method includes receiving an input text and a target image, where the target image including a target face, generating, based on the input text, a sequence of sets of acoustic features corresponding to the input text, generating, based on the sequence of sets of acoustic features, a sequence of sets of mouth key points, generating, based on the sequence of sets of mouth key points, a sequence of sets of facial key points, determining, based on the set sequence of sets of facial key points, a sequence of deformations of the target face, and applying the sequence of deformations to the target image, thereby generating a sequence of frames of an output video.

公开(公告)号：US20230334090A1

公开(公告)日：2023-10-19

申请号：US18139367

申请日：2023-04-26

Applicant: Snap Inc.

Inventor： Jeremy Voss ,  Victor Shaburov ,  Aleksandr Mashrabov ,  Dmitriy Matov ,  Hanna Rulevska ,  Dmytro Ishchenko

IPC: G06F16/732 ,  G06F16/71

CPC classification number: G06F16/7328 ,  G06F16/71 ,  H04L51/10

Abstract: Systems and methods for searching and ranking videos are provided. An example method includes receiving an input from a user via a multimedia messaging application (MMA), selecting, from a database configured to store videos, a subset of videos based on the input and a predetermined rule, rendering, via the MMA, the subset of videos, determining that the user has viewed a video from the subset of videos, determining whether the user has shared, via the MMA, the viewed video, updating a log based on information concerning the subset of videos and the viewed video, and updating, based on the log, a model of the predetermined rule for selecting the videos. The model is trained based on pairs of first videos and second videos, where the first videos have been viewed and shared by the user and the second videos have been viewed but not shared by the user.

公开(公告)号：US11669569B2

公开(公告)日：2023-06-06

申请号：US17581350

申请日：2022-01-21

Applicant: Snap Inc.

Inventor： Jeremy Voss ,  Victor Shaburov ,  Aleksandr Mashrabov ,  Dmitriy Matov ,  Hanna Rulevska ,  Dmytro Ishchenko

IPC: G06F16/732 ,  G06F16/71 ,  H04L51/10

CPC classification number: G06F16/7328 ,  G06F16/71 ,  H04L51/10

Abstract: A system for searching and ranking modifiable videos is provided. In one example embodiment, the system includes a at least one processor and a memory storing processor-executable codes, wherein the at least one processor is configured to implement the following operations upon executing the processor-executable codes: processing a first set of search requests by a first search rule to obtain a first set of first results; processing a second set of search requests by a second search rule to obtain a second set of second results; determining a first metric based, at least partially, on the first set of first results; determining a second metric based at least partially on the second set of second results; selecting, based on the first metric and the second metric, a search rule from the first search rule and the second search rule; and configuring a search engine to apply the search rule.

公开(公告)号：US20220392133A1

公开(公告)日：2022-12-08

申请号：US17881947

申请日：2022-08-05

Applicant: Snap Inc.

Inventor： Yurii Volkov ,  Pavel Savchenkov ,  Nikolai Smirnov ,  Aleksandr Mashrabov

IPC: G06T13/40 ,  G06T7/70 ,  G06V40/16

Abstract: Provided are systems and methods for realistic head turns and face animation synthesis. An example method includes receiving a source frame of a source video, where the source frame includes a head and a face of a source actor, generating source pose parameters corresponding to a pose of the head and a facial expression of the source actor; receiving a target image including a target head and a target face of a target person, determining target identity information associated with the target head and the target face of the target person, replacing source identity information in the source pose parameters with the target identity information to obtain further source pose parameters, and generating an output frame of an output video that includes a modified image of the target face and the target head adopting the pose of the head and the facial expression of the source actor.

公开(公告)号：US11308676B2

公开(公告)日：2022-04-19

申请号：US17062309

申请日：2020-10-02

Applicant: Snap Inc.

Inventor： Egor Nemchinov ,  Sergei Gorbatyuk ,  Aleksandr Mashrabov ,  Egor Spirin ,  Iaroslav Sokolov ,  Andrei Smirdin ,  Igor Tukh

IPC: G06T13/40 ,  G06T7/194 ,  G06N3/08 ,  G06T3/00 ,  G06T15/04 ,  G06T17/20

Abstract: Provided are systems and methods for single image-based body animation. An example method includes receiving an input image that includes a body of a person and segmenting the input image into a body portion and a background portion. The method further includes fitting a model to the body portion. The model is configured to receive a set of pose parameters representing a pose of the body and generate an output image including an image of the body adopting the pose. The method further includes receiving a series of further sets of pose parameters, each representing at least one of further poses of the body. The further sets of pose parameters are generated using a generic model. The method also includes generating a series of output images of the body adopting the further poses and generating an output video based on the series of output images.

公开(公告)号：US20220070291A1

公开(公告)日：2022-03-03

申请号：US17522947

申请日：2021-11-10

Applicant: Snap Inc.

Inventor： Jeremy Voss ,  Victor Shaburov ,  Ivan Babanin ,  Aleksandr Mashrabov ,  Roman Golobokov

IPC: H04M1/72439 ,  H04L12/58 ,  G06T11/60 ,  G06T13/80 ,  H04M1/72427 ,  H04M1/72436 ,  H04M1/72442

Abstract: A system for customizing modifiable videos of a multimedia messaging application (MMA) is provided. In one example embodiment, the system includes at least one processor and a memory storing processor-executable codes, wherein the at least one processor is configured to analyze recent messages of a user to determine a context of the recent messages; determine, based on the context, a customized feature; select, based on the context, a list of relevant modifiable videos from a database configured to store modifiable videos, the modifiable videos being associated with a preset modifiable feature; replace the preset modifiable feature in the relevant modifiable videos with the customized feature; and render a modifiable video from the list of relevant modifiable videos for viewing by the user, the rendering including displaying the customized feature in the relevant modifiable videos.

公开(公告)号：US20210406300A1

公开(公告)日：2021-12-30

申请号：US16990505

申请日：2020-08-11

Applicant: Snap Inc.

Inventor： Aleksandr Mashrabov ,  Victor Shaburov ,  Grigoriy Tkachenko ,  Jeremy Baker Voss

IPC: G06F16/535 ,  G06K9/00 ,  G06F16/54 ,  G06F16/58

Abstract: A personalized preview system to receive a request to access a collection of media items from a user of a user device. Responsive to receiving the request to access the collection of media items, the personalized preview system accesses user profile data associated with the user, wherein the user profile data includes an image. For example, the image may comprise a depiction of a face, wherein the face comprises a set of facial landmarks. Based on the image, the personalized preview system generates one or more media previews based on corresponding media templates and the image, and displays the one or more media previews within a presentation of the collection of media items at a client device of the user.

公开(公告)号：US20210327404A1

公开(公告)日：2021-10-21

申请号：US17355834

申请日：2021-06-23

Applicant: Snap Inc.

Inventor： Pavel Savchenkov ,  Maxim Lukin ,  Aleksandr Mashrabov

IPC: G10L13/00 ,  G06K9/00 ,  G10L13/08 ,  G06T13/40

Abstract: Provided are systems and methods for text and audio-based real-time face reenactment. An example method includes receiving an input text and a target image, the target image including a target face; generating, based on the input text, a sequence of sets of acoustic features representing the input text; generating, based on the sequence of sets of acoustic features, a sequence of sets of mouth key points; generating, based on the sequence of sets of mouth key points, a sequence of sets of facial key points; generating, by the computing device and based on the sequence of sets of the facial key points and the target image, a sequence of frames; and generating, based on the sequence of frames, an output video. Each of the frames includes the target face modified based on at least one set of mouth key points of the sequence of sets of mouth key points.

公开(公告)号：US20210303622A1

公开(公告)日：2021-09-30

申请号：US16835656

申请日：2020-03-31

Applicant: Snap Inc.

Inventor： Jeremy Voss ,  Victor Shaburov ,  Aleksandr Mashrabov ,  Dmitriy Matov ,  Hanna Rulevska ,  Dmytro Ishchenko

IPC: G06F16/732 ,  G06F16/71

Abstract: A system for searching and ranking modifiable videos in a multimedia messaging application (MMA) is provided. In one example embodiment, the system includes a database configured to store modifiable videos, the modifiable videos being associated with text messages and rankings, a processor, and a memory storing processor-executable codes, wherein the processor is configured to implement the following operations upon executing the processor-executable codes: receiving, via the MMA, an input of a user; selecting, based on the input, a list of relevant modifiable videos from the database; rendering, via the MMA, the list of relevant modifiable videos for viewing by the user; determining that the user has shared, via the MMA, a modifiable video from the list; storing an information concerning the list and the shared modifiable video into a statistical log; and updating, based on the information in the statistical log, the rankings of the modifiable videos in the database.