-
公开(公告)号：US20230316666A1
公开(公告)日：2023-10-05
申请号：US17842006
申请日：2022-06-16
Applicant: Snap Inc.
Inventor： Madiyar Aitbayev , Brian Fulkerson , Riza Alp Guler , Georgios Papandreou , Himmy Tam
CPC classification number: G06T19/006 , G06V10/70 , G06V10/26 , G06V20/20 , G06N20/20
Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; extracting a portion of the image; applying a first machine learning model stage to the portion to predict a depth of a point of interest for the data representing the depiction of the person; applying a second machine learning model stage to the portion of the image to predict a relative depth of each pixel in the portion of the image to the predicted depth of the point of interest; generating dense depth reconstruction of the data representing the depiction of the person based on outputs of the first and second stages of the machine learning model; and applying one or more AR elements to the image based on the dense depth reconstruction.
-
公开(公告)号：US20230316665A1
公开(公告)日：2023-10-05
申请号：US17841994
申请日：2022-06-16
Applicant: Snap Inc.
Inventor： Madiyar Aitbayev , Brian Fulkerson , Riza Alp Guler , Georgios Papandreou , Himmy Tam
CPC classification number: G06T19/006 , G06T7/11 , G06T7/194 , G06T15/506 , G06T7/70 , G06T19/20 , G06T15/60 , G06T2207/30196 , G06T2207/20021 , G06T2207/20081 , G06T2207/10016 , G06T2219/2012 , G06T2207/20084 , G06T2200/08 , G06T2210/16
Abstract: Methods and systems are disclosed for performing operations for applying augmented reality elements to a person depicted in an image. The operations include receiving an image that includes data representing a depiction of a person; generating a segmentation of the data representing the person depicted in the image; extracting a portion of the image corresponding to the segmentation of the data representing the person depicted in the image; applying a machine learning model to the portion of the image to predict a surface normal tensor for the data representing the depiction of the person, the surface normal tensor representing surface normals of each pixel within the portion of the image; and applying one or more augmented reality (AR) elements to the image based on the surface normal tensor.
-
公开(公告)号：US20230090645A1
公开(公告)日：2023-03-23
申请号：US17448158
申请日：2021-09-20
Applicant: Snap Inc
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Riza Alp Guler
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.
-
公开(公告)号：US20230080584A1
公开(公告)日：2023-03-16
申请号：US17447614
申请日：2021-09-14
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; obtaining an external mesh associated with an augmented reality (AR) element; automatically establishing a correspondence between a portion of the 3D body mesh associated with the real-world object and the external mesh; blending the portion of the 3D body mesh into the external mesh according to a blending parameter; and as the portion of the 3D body mesh is being blended into the external mesh, modifying the video to depict a portion of the real-world object being changed into the AR element.
-
公开(公告)号：US20250131584A1
公开(公告)日：2025-04-24
申请号：US19006543
申请日：2024-12-31
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Itamar Berger
IPC: G06T7/70 , G06F18/214 , G06F18/24 , G06T19/00 , G06V40/10
Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for performing operations comprising: receiving a video that depicts a person. The operations further include identifying a set of skeletal joints of the person. The operations further include identifying a pose of the person depicted in the video based on positioning of the set of skeletal joints (or detecting a hand pose, detecting a mirror frame, or detecting a mobile device). The operations further include determining, based on the pose of the person (or detecting a hand pose, detecting a mirror frame, or detecting a mobile device), that the video comprises a mirror reflection of the person. The operations further include, in response to determining that the video comprises the mirror reflection of the person, causing display of a 3D virtual object in the video.
-
Patent Agency Ranking
公开(公告)号：US20250095308A1
公开(公告)日：2025-03-20
申请号：US18969689
申请日：2024-12-05
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Riza Alp Guler
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.
-
公开(公告)号：US12086946B2
公开(公告)日：2024-09-10
申请号：US18475813
申请日：2023-09-27
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson
CPC classification number: G06T19/006 , G06T7/251 , G06T13/40 , G06T13/80 , G06T17/20 , G06T19/20 , G06T2200/24 , G06T2207/10016 , G06T2219/2021
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; obtaining an external mesh associated with an augmented reality (AR) element; automatically establishing a correspondence between a portion of the 3D body mesh associated with the real-world object and the external mesh; blending the portion of the 3D body mesh into the external mesh according to a blending parameter; and as the portion of the 3D body mesh is being blended into the external mesh, modifying the video to depict a portion of the real-world object being changed into the AR element.
-
公开(公告)号：US20240070986A1
公开(公告)日：2024-02-29
申请号：US18502634
申请日：2023-11-06
Applicant: Snap Inc.
Inventor： Matan Zohar , Artem Bondich , Yanli Zhao , Brian Fulkerson
CPC classification number: G06T17/205 , G06T7/251 , G06T7/70 , G06T19/006 , G06T2207/10016
Abstract: Methods and systems are disclosed for performing operations for deforming an external mesh. The operations comprise receiving a video that includes a depiction of a real-world object. The operations comprise generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video. The operations comprise obtaining an external mesh associated with an Augmented-Reality (AR) element. The operations comprise accessing a plurality of deformation attributes associated with the external mesh, each attribute corresponding to a different deformation model. The operations comprise separately deforming, based on respective deformation models, a first portion of the external mesh and a second portion of the external mesh. The operations comprise modifying the video to include a display of the AR element based on the separately deformed first and second portions of the external mesh.
-
公开(公告)号：US11836866B2
公开(公告)日：2023-12-05
申请号：US17448158
申请日：2021-09-20
Applicant: Snap Inc.
Inventor： Matan Zohar , Yanli Zhao , Brian Fulkerson , Riza Alp Guler
CPC classification number: G06T19/006 , G06T7/20 , G06T7/60 , G06T11/001 , G06T17/20 , G06T2219/2016 , G06T2219/2021
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; determining UV positions of the real-world object depicted in the video to obtain pixel values associated with the UV positions; generating an external mesh and associated augmented reality (AR) element representing the real-world object based on the pixel values associated with the UV positions; deforming the external mesh based on changes to the 3D body mesh and a deformation parameter; and modifying the video to replace the real-world object with the AR element based on the deformed external mesh.
-
公开(公告)号：US20230115531A1
公开(公告)日：2023-04-13
申请号：US17498394
申请日：2021-10-11
Applicant: Snap Inc.
Inventor： Matan Zohar , Artem Bondich , Yanli Zhao , Brian Fulkerson
Abstract: Methods and systems are disclosed for performing operations for deforming an external mesh. The operations comprise receiving a video that includes a depiction of a real-world object. The operations comprise generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video. The operations comprise obtaining an external mesh associated with an Augmented-Reality (AR) element. The operations comprise accessing a plurality of deformation attributes associated with the external mesh, each attribute corresponding to a different deformation model. The operations comprise separately deforming, based on respective deformation models, a first portion of the external mesh and a second portion of the external mesh. The operations comprise modifying the video to include a display of the AR element based on the separately deformed first and second portions of the external mesh.
-
-
-
-
-
-
-
-
-
Search Results
32
records
(took 0.022 seconds)
