公开(公告)号：US20210335004A1

公开(公告)日：2021-10-28

申请号：US15929333

申请日：2020-04-27

Applicant: Snap Inc.

Inventor： Matan Zohar ,  Avihay Assouline ,  Gal Dudovitch ,  Itamar Berger

IPC: G06T7/73 ,  G06T3/40 ,  G06T11/60

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising: receiving a monocular image that includes a depiction of a body of a user in a given pose; detecting a plurality of skeletal joints of the body depicted in the monocular image; generating an outline of the body of the user based on the plurality of skeletal joints; retrieving a mask that represents a target pose; comparing the outline of the body depicted in the monocular image with the mask that represents the target pose; and determining that the given pose of the body of the user matches the target pose in response to comparing the outline of the body with the mask.

公开(公告)号：US20210209825A1

公开(公告)日：2021-07-08

申请号：US17212555

申请日：2021-03-25

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger ,  Yuncheng Li

IPC: G06T13/40 ,  G06K9/00

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the monocular image; and determining a pose represented by the body depicted in the monocular image based on the detected plurality of skeletal joints of the body. A pose of an avatar is modified to match the pose represented by the body depicted in the monocular image by adjusting a set of skeletal joints of a rig of an avatar based on the detected plurality of skeletal joints of the body; and the avatar having the modified pose that matches the pose represented by the body depicted in the monocular image is generated for display.

公开(公告)号：US20210182555A1

公开(公告)日：2021-06-17

申请号：US16710980

申请日：2019-12-11

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger ,  Gal Dudovitch ,  Matan Zohar

IPC: G06K9/00 ,  H04N21/4402 ,  G06K9/62

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body based on the monocular image; accessing a video feed comprising a plurality of monocular images received prior to the monocular image; filtering, using the video feed, the plurality of skeletal joints of the body detected based on the monocular image; and determining a pose represented by the body depicted in the monocular image based on the filtered plurality of skeletal joints of the body.

公开(公告)号：US11036989B1

公开(公告)日：2021-06-15

申请号：US16710980

申请日：2019-12-11

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger ,  Gal Dudovitch ,  Matan Zohar

IPC: G06T7/55 ,  G06K9/00 ,  H04N21/4402 ,  G06K9/62 ,  H04L12/58

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body based on the monocular image; accessing a video feed comprising a plurality of monocular images received prior to the monocular image; filtering, using the video feed, the plurality of skeletal joints of the body detected based on the monocular image; and determining a pose represented by the body depicted in the monocular image based on the filtered plurality of skeletal joints of the body.

公开(公告)号：US20200250874A1

公开(公告)日：2020-08-06

申请号：US16269312

申请日：2019-02-06

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger ,  Yuncheng Li

IPC: G06T13/40 ,  G06K9/00

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the monocular image; and determining a pose represented by the body depicted in the monocular image based on the detected plurality of skeletal joints of the body. A pose of an avatar is modified to match the pose represented by the body depicted in the monocular image by adjusting a set of skeletal joints of a rig of an avatar based on the detected plurality of skeletal joints of the body; and the avatar having the modified pose that matches the pose represented by the body depicted in the monocular image is generated for display.

公开(公告)号：US20250139813A1

公开(公告)日：2025-05-01

申请号：US19010631

申请日：2025-01-06

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger ,  Nir Malbin ,  Gal Sasson

IPC: G06T7/70 ,  G06T5/77 ,  G06T7/10 ,  G06T7/20 ,  G06T11/00 ,  G06T19/00 ,  G06T19/20 ,  G06V10/54 ,  G06V10/56

Abstract: Methods and systems are disclosed for performing operations for transferring garments in a video from one real-world object to another in real time. The operations comprise receiving a first video that includes a depiction of a first person wearing a first garment in a first pose and obtaining a second video that includes a depiction of a second person wearing a second garment in a second pose. The operations comprise modifying a pose of the second person to match the first pose of the first person depicted in the first video. The operations comprise generating a whole-body segmentation of the second garment which the second person is wearing in the second video and changing an appearance of the first person from wearing the first garment to wearing the second garment based on the whole-body segmentation of the second garment which the second person is wearing in the second video.

公开(公告)号：US20250131584A1

公开(公告)日：2025-04-24

申请号：US19006543

申请日：2024-12-31

Applicant: Snap Inc.

Inventor： Matan Zohar ,  Yanli Zhao ,  Brian Fulkerson ,  Itamar Berger

IPC: G06T7/70 ,  G06F18/214 ,  G06F18/24 ,  G06T19/00 ,  G06V40/10

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising: receiving a video that depicts a person. The operations further include identifying a set of skeletal joints of the person. The operations further include identifying a pose of the person depicted in the video based on positioning of the set of skeletal joints (or detecting a hand pose, detecting a mirror frame, or detecting a mobile device). The operations further include determining, based on the pose of the person (or detecting a hand pose, detecting a mirror frame, or detecting a mobile device), that the video comprises a mirror reflection of the person. The operations further include, in response to determining that the video comprises the mirror reflection of the person, causing display of a 3D virtual object in the video.

公开(公告)号：US20250095198A1

公开(公告)日：2025-03-20

申请号：US18965798

申请日：2024-12-02

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger ,  Gal Dudovitch ,  Matan Zohar

IPC: G06T7/73 ,  G06F18/214 ,  G06V20/20 ,  G06V20/40 ,  G06V20/64 ,  G06V40/10 ,  G06V40/20 ,  H04L51/04 ,  H04N21/4402

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body based on the monocular image; accessing a video feed comprising a plurality of monocular images received prior to the monocular image; filtering, using the video feed, the plurality of skeletal joints of the body detected based on the monocular image; and determining a pose represented by the body depicted in the monocular image based on the filtered plurality of skeletal joints of the body.

公开(公告)号：US20250078127A1

公开(公告)日：2025-03-06

申请号：US18458807

申请日：2023-08-30

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger

IPC: G06Q30/0601 ,  G06V10/26 ,  G06V10/764 ,  G06V40/10

Abstract: Methods and systems are disclosed for using machine learning models to recommend fashion item fit styles based on body surface landmarks. The methods and systems access one or more images depicting a person wearing one or more fashion items and process, using one or more machine learning models, the one or more images to estimate a data set comprising a set of body landmarks of the person, a set of garment classifications associated with the one or more fashion items, and a set of garment segmentations for the one or more fashion items. The methods and systems identify one or more fit styles associated with the person based on the estimated data set and cause presentation of one or more real-world fashion items matching the identified one or more fit styles.

公开(公告)号：US20250069312A1

公开(公告)日：2025-02-27

申请号：US18943660

申请日：2024-11-11

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger ,  Gal Dudovitch ,  Matan Zohar

IPC: G06T13/40 ,  G06F3/04817 ,  G06F3/0488 ,  G06T7/20 ,  G06T15/20 ,  G06T19/00

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising: capturing a video that depicts a person; identifying a set of skeletal joints of the person depicted in the video; storing a movement vector representing previously captured three-dimensional (3D) movement of the set of skeletal joints of the person depicted in the video; receiving input that selects a 3D avatar; and animating, based on the movement vector, the 3D avatar to mimic the previously captured 3D movement of the set of skeletal joints of the person depicted in the video.