公开(公告)号：US20210209825A1

公开(公告)日：2021-07-08

申请号：US17212555

申请日：2021-03-25

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger ,  Yuncheng Li

IPC: G06T13/40 ,  G06K9/00

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the monocular image; and determining a pose represented by the body depicted in the monocular image based on the detected plurality of skeletal joints of the body. A pose of an avatar is modified to match the pose represented by the body depicted in the monocular image by adjusting a set of skeletal joints of a rig of an avatar based on the detected plurality of skeletal joints of the body; and the avatar having the modified pose that matches the pose represented by the body depicted in the monocular image is generated for display.

公开(公告)号：US10796482B2

公开(公告)日：2020-10-06

申请号：US16210927

申请日：2018-12-05

Applicant: Snap Inc.

Inventor： Liuhao Ge ,  Zhou Ren ,  Yuncheng Li ,  Zehao Xue ,  Yingying Wang

IPC: G06T17/20 ,  G06T7/73 ,  G06N3/08 ,  G06N3/04

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for receiving a monocular image that includes a depiction of a hand and extracting features of the monocular image using a plurality of machine learning techniques. The program and method further include modeling, based on the extracted features, a pose of the hand depicted in the monocular image by adjusting skeletal joint positions of a three-dimensional (3D) hand mesh using a trained graph convolutional neural network (CNN); modeling, based on the extracted features, a shape of the hand in the monocular image by adjusting blend shape values of the 3D hand mesh representing surface features of the hand depicted in the monocular image using the trained graph CNN; and generating, for display, the 3D hand mesh adjusted to model the pose and shape of the hand depicted in the monocular image.

公开(公告)号：US20200250874A1

公开(公告)日：2020-08-06

申请号：US16269312

申请日：2019-02-06

Applicant: Snap Inc.

Inventor： Avihay Assouline ,  Itamar Berger ,  Yuncheng Li

IPC: G06T13/40 ,  G06K9/00

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body depicted in the monocular image; and determining a pose represented by the body depicted in the monocular image based on the detected plurality of skeletal joints of the body. A pose of an avatar is modified to match the pose represented by the body depicted in the monocular image by adjusting a set of skeletal joints of a rig of an avatar based on the detected plurality of skeletal joints of the body; and the avatar having the modified pose that matches the pose represented by the body depicted in the monocular image is generated for display.

公开(公告)号：US10713754B1

公开(公告)日：2020-07-14

申请号：US15908461

申请日：2018-02-28

Applicant: Snap Inc.

Inventor： Guohui Wang ,  Sumant Milind Hanumante ,  Ning Xu ,  Yuncheng Li

IPC: G06K9/62 ,  G06T3/40 ,  G06N3/08 ,  G06N3/04 ,  G06T1/20 ,  G06T11/60 ,  G06N3/063

Abstract: Remote distribution of multiple neural network models to various client devices over a network can be implemented by identifying a native neural network and remotely converting the native neural network to a target neural network based on a given client device operating environment. The native neural network can be configured for execution using efficient parameters, and the target neural network can use less efficient but more precise parameters.

公开(公告)号：US12289283B2

公开(公告)日：2025-04-29

申请号：US18150001

申请日：2023-01-04

Applicant: Snap Inc.

Inventor： Harsh Agrawal ,  Xuan Huang ,  Jung Hyun Kim ,  Yuncheng Li ,  Yiwei Ma ,  Tao Ning ,  Ye Tao

IPC: H04L51/52 ,  G06F3/0482 ,  G06T7/00 ,  H04L51/222 ,  H04L65/1069 ,  H04L67/01

Abstract: Systems, methods, devices, computer readable instruction media, and other embodiments are described for automated image processing and insight presentation. One embodiment involves receiving a plurality of ephemeral content messages from a plurality of client devices, and processing the messages to identify content associated with at least a first content type. A set of analysis data associated with the first content type is then generated from the messages, and portions of the messages associated with the first content type are processed to generate a first content collection. The first content collection and the set of analysis data are then communicated to a client device configured for a display interface comprising the first content collection and a representation of at least a portion of the set of analysis data.

公开(公告)号：US12165335B2

公开(公告)日：2024-12-10

申请号：US18460335

申请日：2023-09-01

Applicant: Snap Inc.

Inventor： Yuncheng Li ,  Linjie Luo ,  Xuecheng Nie ,  Ning Zhang

IPC: G06T7/246 ,  G06T7/73 ,  G06V10/764 ,  G06V10/82 ,  G06V20/40 ,  G06V40/20 ,  G06F3/04817 ,  H04L51/04 ,  H04L67/01

Abstract: Systems, devices, media and methods are presented for a human pose tracking framework. The human pose tracking framework may identify a message with video frames, generate, using a composite convolutional neural network, joint data representing joint locations of a human depicted in the video frames, the generating of the joint data by the composite convolutional neural network done by a deep convolutional neural network operating on one portion of the video frames, a shallow convolutional neural network operating on a another portion of the video frames, and tracking the joint locations using a one-shot learner neural network that is trained to track the joint locations based on a concatenation of feature maps and a convolutional pose machine. The human pose tracking framework may store, the joint locations, and cause presentation of a rendition of the joint locations on a user interface of a client device.

公开(公告)号：US11847760B2

公开(公告)日：2023-12-19

申请号：US17714764

申请日：2022-04-06

Applicant: Snap Inc.

Inventor： Guohui Wang ,  Sumant Milind Hanumante ,  Ning Xu ,  Yuncheng Li

IPC: G06N3/08 ,  G06T3/40 ,  G06N3/04 ,  G06T1/20 ,  G06T11/60 ,  G06N3/063

CPC classification number: G06T3/4046 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08 ,  G06T1/20 ,  G06T11/60 ,  G06T2207/20081

Abstract: Remote distribution of multiple neural network models to various client devices over a network can be implemented by identifying a native neural network and remotely converting the native neural network to a target neural network based on a given client device operating environment. The native neural network can be configured for execution using efficient parameters, and the target neural network can use less efficient but more precise parameters.

公开(公告)号：US11830209B2

公开(公告)日：2023-11-28

申请号：US17651524

申请日：2022-02-17

Applicant: Snap Inc.

Inventor： Travis Chen ,  Samuel Edward Hare ,  Yuncheng Li ,  Tony Mathew ,  Jonathan Solichin ,  Jianchao Yang ,  Ning Zhang

IPC: G06T7/50 ,  G06T19/20 ,  G06T7/20 ,  G06T19/00 ,  G06T7/73 ,  G06V10/20 ,  G06V20/20 ,  G06V20/40 ,  G06V10/764 ,  G06V20/64

CPC classification number: G06T7/50 ,  G06T7/20 ,  G06T7/73 ,  G06T19/006 ,  G06T19/20 ,  G06V10/255 ,  G06V10/764 ,  G06V20/20 ,  G06V20/40 ,  G06V20/64 ,  G06T2207/10016 ,  G06T2207/20084 ,  G06T2210/12 ,  G06T2219/2016

Abstract: Systems, devices, media, and methods are presented for object detection and inserting graphical elements into an image stream in response to detecting the object. The systems and methods detect an object of interest in received frames of a video stream. The systems and methods identify a bounding box for the object of interest and estimate a three-dimensional position of the object of interest based on a scale of the object of interest. The systems and methods generate one or more graphical elements having a size based on the scale of the object of interest and a position based on the three-dimensional position estimated for the object of interest. The one or more graphical elements are generated within the video stream to form a modified video stream. The systems and methods cause presentation of the modified video stream including the object of interest and the one or more graphical elements.

公开(公告)号：US20230376757A1

公开(公告)日：2023-11-23

申请号：US18230499

申请日：2023-08-04

Applicant: Snap Inc.

Inventor： Yuncheng Li ,  Zhou Ren ,  Ning Xu ,  Enxu Yan ,  Tan Yu

IPC: G06N3/08 ,  G06T7/55 ,  G06T7/33 ,  G06V20/64 ,  G06F18/214 ,  G06F18/2431 ,  G06V10/82 ,  G06V10/44 ,  G06V20/20 ,  G06V20/40

CPC classification number: G06N3/08 ,  G06T7/55 ,  G06T7/344 ,  G06V20/64 ,  G06F18/214 ,  G06F18/2431 ,  G06V10/82 ,  G06V10/454 ,  G06V20/20 ,  G06V20/41

Abstract: Systems and methods are disclosed for capturing multiple sequences of views of a three-dimensional object using a plurality of virtual cameras. The systems and methods generate aligned sequences from the multiple sequences based on an arrangement of the plurality of virtual cameras in relation to the three-dimensional object. Using a convolutional network, the systems and methods classify the three-dimensional object based on the aligned sequences and identify the three-dimensional object using the classification.

公开(公告)号：US11783494B2

公开(公告)日：2023-10-10

申请号：US17660462

申请日：2022-04-25

Applicant: Snap Inc.

Inventor： Yuncheng Li ,  Linjie Luo ,  Xuecheng Nie ,  Ning Zhang

IPC: G06T7/246 ,  G06T7/73 ,  G06V20/40 ,  G06V10/764 ,  G06V10/82 ,  G06V40/20 ,  G06F3/04817 ,  H04L51/04 ,  H04L67/01

CPC classification number: G06T7/246 ,  G06T7/73 ,  G06V10/764 ,  G06V10/82 ,  G06V20/46 ,  G06V40/23 ,  G06F3/04817 ,  G06T2200/24 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30196 ,  H04L51/04 ,  H04L67/01

Abstract: Systems, devices, media and methods are presented for a human pose tracking framework. The human pose tracking framework may identify a message with video frames, generate, using a composite convolutional neural network, joint data representing joint locations of a human depicted in the video frames, the generating of the joint data by the composite convolutional neural network done by a deep convolutional neural network operating on one portion of the video frames, a shallow convolutional neural network operating on a another portion of the video frames, and tracking the joint locations using a one-shot learner neural network that is trained to track the joint locations based on a concatenation of feature maps and a convolutional pose machine. The human pose tracking framework may store, the joint locations, and cause presentation of a rendition of the joint locations on a user interface of a client device.