公开(公告)号：US20250148816A1

公开(公告)日：2025-05-08

申请号：US18502868

申请日：2023-11-06

Applicant: Snap Inc.

Inventor： Maksim Gusarov ,  Kwot Sin Lee ,  Patrick Poirson ,  Chen Wang

IPC: G06V20/70 ,  G06F40/40 ,  G06N3/0455 ,  G06T11/00 ,  G06V10/774 ,  G06V20/20 ,  G06V20/40

Abstract: A second input image is generated by applying a target augmented reality (AR) effect to a first input image. The first input image and the second input image are provided to a first visual-semantic machine learning model to obtain output describing at least one feature of the target AR effect. The first visual-semantic machine learning model is fine-tuned from a second visual-semantic machine learning model by using training samples. Each training sample comprises a first training image, a second training image, and a training description of a given AR effect. The second training image is generated by applying the given AR effect to the first training image. A description of the target AR effect is selected based on the output of the visual-semantic machine learning model. The description of the target AR effect is stored in association with an identifier of the target AR effect.

公开(公告)号：US20250148218A1

公开(公告)日：2025-05-08

申请号：US18502679

申请日：2023-11-06

Applicant: Snap Inc.

Inventor： Maksim Gusarov ,  Kwot Sin Lee ,  Yanjia Li ,  Patrick Poirson ,  Chen Wang

IPC: G06F40/40 ,  G06T19/00 ,  G06V10/72

Abstract: A first image and a second image are accessed. The second image is generated by applying an augmented reality (AR) effect to the first image. The first image, the second image, and a prompt are provided to a visual-semantic machine learning model to obtain output describing at least one feature of the AR effect. A description of the AR effect is generated based on the output of the visual-semantic machine learning model. The description of the AR effect is stored in association with an identifier of the AR effect.

公开(公告)号：US20240355063A1

公开(公告)日：2024-10-24

申请号：US18304078

申请日：2023-04-20

Applicant: Snap Inc.

Inventor： Zhenpeng Zhou ,  Patrick Poirson ,  Maksim Gusarov ,  Chen Wang ,  Oleg Tovstyi

IPC: G06T19/00 ,  G06T1/00 ,  G06V10/74 ,  H04N5/262

CPC classification number: G06T19/006 ,  G06T1/0021 ,  G06V10/761 ,  H04N5/2621

Abstract: An input video item that includes a target visual augmentation is accessed. A machine learning model uses the input video item to generate an embedding. The embedding may comprise a vector representation of a visual effect of the target visual augmentation. The machine learning model is trained, in an unsupervised training phase, to minimize loss between training video representations generated within each of a plurality of training sets. Each training set comprises a plurality of different training video items that each include a predefined visual augmentation. Based on the generation of the embedding of the input video item, the target visual augmentation is mapped to an augmentation identifier.

公开(公告)号：US20240087286A1

公开(公告)日：2024-03-14

申请号：US17939256

申请日：2022-09-07

Applicant: Snap Inc.

Inventor： Huseyin Coskun ,  Alireza Zareian ,  Joshua Moore ,  Chen Wang

IPC: G06V10/762 ,  G06T5/50 ,  G06V10/82

CPC classification number: G06V10/762 ,  G06T5/50 ,  G06V10/82 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20224

Abstract: Systems and methods are provided for clustering videos. The system accesses a plurality of content items, the plurality of content items comprising a first set of RGB video frames and a second set of optical flow frames corresponding to the first set of RGB video frames. The system processes the first set of RGB video frames by a first machine learning model to generate a first optimal assignment for the first set of RGB video frames, the first optimal assignment representing initial clustering of the first set of RGB video frames. The system generates an updated first optimal assignment for the first set of RGB video frames based on the first optimal assignment for the first set of RGB video frames and a second optimal assignment of the second set of optical flow frames, the second optimal assignment representing initial clustering of the second set of optical flow frames.