-
公开(公告)号:US20240307783A1
公开(公告)日:2024-09-19
申请号:US18121268
申请日:2023-03-14
Applicant: Snap Inc.
Inventor: Willi Menapace , Aliaksandr Siarohin , Sergey Tulyakov
Abstract: A framework trains game-engine-like neural models from annotated videos to generate a Learnable Game Engine (LGE) that maintains states of the scene, objects and agents in it, and enables rendering the environment from a controllable viewpoint. The LGE models the logic of the game and the rules of physics, making it possible for the user to play the game by specifying both high- and low-level action sequences. The LGE also unlocks a director's mode where the game is played by plotting behind the scenes, specifying high-level actions and goals for the agents using text-based instructions. To implement the director's mode, a trained diffusion-based animation model navigates the scene using high-level constraints, to enable play against an adversary, and to devise the strategy to win a point. To render the resulting state of the environment and its agents, a compositional neural radiance field (NeRF) representation is used in a synthesis model.
-
公开(公告)号:US12094135B2
公开(公告)日:2024-09-17
申请号:US17829644
申请日:2022-06-01
Applicant: Snap Inc.
Inventor: Sergey Tulyakov , Roman Furko , Aleksei Stoliar
IPC: G06T7/277 , G06F18/214 , G06T7/246 , G06T7/73 , G06T11/60 , G06V10/25 , G06V10/62 , G06V10/764 , G06V10/774 , G06V10/776 , G06V10/82 , G06V20/20 , G06V20/40 , G06V40/10 , G06V40/16
CPC classification number: G06T7/277 , G06F18/2148 , G06F18/2155 , G06T7/246 , G06T7/73 , G06T11/60 , G06V10/25 , G06V10/764 , G06V10/7747 , G06V10/7753 , G06V10/776 , G06V10/82 , G06V20/20 , G06V20/40 , G06V40/10 , G06V40/168 , G06T2207/10016 , G06T2207/20081 , G06T2207/20084 , G06T2207/20132 , G06V10/62 , G06V2201/07
Abstract: A landmark detection system can more accurately detect landmarks in images using a detection scheme that penalizes for dispersion parameters, such as variance or scale. The landmark detection system can be trained using both labeled and unlabeled training data in a semi-supervised approach. The landmark detection system can further implement tracking of an object across multiple images using landmark data.
-
公开(公告)号:US20240305858A1
公开(公告)日:2024-09-12
申请号:US18118762
申请日:2023-03-08
Applicant: Snap Inc.
Inventor: Dong Tian , Guanshu Chen
IPC: H04N21/433 , G06F3/04883 , H04N19/177 , H04N19/426
CPC classification number: H04N21/4331 , G06F3/04883 , H04N19/177 , H04N19/426
Abstract: A media player providing real time rewind playback of a played media file having segments of frames. A last segment N of the played media file is cached and rendered on a device, such as a mobile device, then a previous segment N−1 is cached and rendered, and the process continues until there are no more segments of the played media file to cache and render. Only a segment of the played media file is cached at a time, rather than the whole media file, such that the played media file can be replayed on the fly.
-
公开(公告)号:US20240305672A1
公开(公告)日:2024-09-12
申请号:US18670245
申请日:2024-05-21
Applicant: Snap Inc.
Inventor: Nathan Kenneth Boyd , Laurent Desserrey , Alex Edelsburg , Benjamin Ralph Hollis , David Paliwoda , Jeremy Baker Voss
IPC: H04L65/1059 , G06F3/0484 , H04L65/75
CPC classification number: H04L65/1059 , G06F3/0484 , H04L65/75
Abstract: Examples relate to a method of operating a web-based video calling system. The method includes detecting the initiation of a web calling session from a web calling client executing within a web browser on a user device. Responsive to this detection, the method involves causing the display of a video feed from a digital image sensor of the camera of the user device. A user interface (UI) is provided that includes interactive elements to facilitate control over the video calling session. The UI is adapted based on predefined conditions to maintain privacy. Specifically, the UI is adapted by automatically obscuring the video feed upon detecting user inactivity or the removal of foreground focus on the display of the user device. This approach ensures enhanced privacy and user control during web-based video calling sessions.
-
公开(公告)号:US20240303945A1
公开(公告)日:2024-09-12
申请号:US18665388
申请日:2024-05-15
Applicant: Snap Inc.
Inventor: Rastan Boroujerdi , Michael John Evans , Panayoti Haritatos
IPC: G06T19/00 , G01C21/36 , G06F3/04815 , G06F3/04842 , H04L9/40
CPC classification number: G06T19/006 , G01C21/3647 , G06F3/04815 , G06F3/04842 , H04L63/083 , G06T2200/24 , G06T2219/024
Abstract: Methods and systems are disclosed for generating AR experiences. The methods and systems access a first component of a plurality of components implemented by the messaging application, the plurality of components comprising an AR experience, each of the plurality of components being configured to be separately launched by the messaging application. The methods and systems store a first state of the first component in a data structure that is shared across the plurality of components; launching. The methods and system launch, by the messaging application, a second component of the plurality of components in response to determining that an interaction has been performed using the first component; and configure a second state of the second component based on the interaction that has been performed using the first component.
-
公开(公告)号:US20240303926A1
公开(公告)日:2024-09-12
申请号:US18179717
申请日:2023-03-07
Applicant: Snap Inc.
Inventor: Riza Alp Guler , Dominik Kulon , Himmy Tam , Haoyang Wang
CPC classification number: G06T17/20 , G06T7/40 , G06T11/00 , G06T2200/24 , G06T2207/10028 , G06T2207/20081 , G06T2207/30196 , G06T2210/22 , G06T2210/56
Abstract: An system for augmenting images using hand surface normal estimation is provided. In a model training phase, 3D models of hands are generated using 3D data of hands in a variety of positions. Target normal training data is generated that includes normals of surfaces of the 3D models and synthetic 2D image training data corresponding to the 3D models and the normals. The target normal training data and the synthetic image training data are used to train a normal estimation model. The normal estimation is used by an interactive application to generate augmentations that are applied to hand image data.
-
公开(公告)号:US12088544B2
公开(公告)日:2024-09-10
申请号:US17991539
申请日:2022-11-21
Applicant: Snap Inc.
Inventor: Nathan Kenneth Boyd , Jacob Catalano , Jeremy Baker Voss
IPC: H04L51/10 , H04L51/063 , H04L51/216
CPC classification number: H04L51/10 , H04L51/063 , H04L51/216
Abstract: Systems and methods herein save ephemeral media content to a conversation thread between two users who are bidirectional connections on a social media platform. The systems and methods receive, on a first device associated with a first user, media content item from a second device associated with a second user, display the received media content item on the first user device, and while causing display of the received media content item, receive a selection from the first user device. In response to receiving the selection, the systems and methods save the media content item to a conversation thread between the first user and the second user and display an indication that the media content item is saved within the conversation thread.
-
公开(公告)号:US12086946B2
公开(公告)日:2024-09-10
申请号:US18475813
申请日:2023-09-27
Applicant: Snap Inc.
Inventor: Matan Zohar , Yanli Zhao , Brian Fulkerson
CPC classification number: G06T19/006 , G06T7/251 , G06T13/40 , G06T13/80 , G06T17/20 , G06T19/20 , G06T2200/24 , G06T2207/10016 , G06T2219/2021
Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; obtaining an external mesh associated with an augmented reality (AR) element; automatically establishing a correspondence between a portion of the 3D body mesh associated with the real-world object and the external mesh; blending the portion of the 3D body mesh into the external mesh according to a blending parameter; and as the portion of the 3D body mesh is being blended into the external mesh, modifying the video to depict a portion of the real-world object being changed into the AR element.
-
公开(公告)号:US12082768B2
公开(公告)日:2024-09-10
申请号:US17513355
申请日:2021-10-28
Applicant: Snap Inc.
Inventor: Dylan Shane Eirinberg , Daniel Trinh
IPC: A47L9/28 , A47L7/00 , G05D1/00 , G06V20/10 , H04L51/10 , H04L51/222 , H04L67/306
CPC classification number: A47L9/2894 , A47L7/0004 , A47L9/2826 , A47L9/2847 , A47L9/2852 , G05D1/0016 , G05D1/0221 , G05D1/0287 , G06V20/176 , H04L51/10 , H04L51/222 , H04L67/306 , A47L2201/04 , A47L2201/06
Abstract: Methods and systems are disclosed for controlling connected devices. For example, a messaging application implemented on a client device detects a real-world object depicted in a received image and determines a current location of the client device. The messaging application identifies a plurality of robotic cleaning devices associated with an account of the messaging application. The messaging application transmits a message comprising the current location of the client device to a first robotic cleaning device of the plurality of robotic cleaning devices and causes the first robotic cleaning device to clean the real-world object depicted in the received image based on the message transmitted by the messaging application.
-
公开(公告)号:US20240296645A1
公开(公告)日:2024-09-05
申请号:US18664596
申请日:2024-05-15
Applicant: Snap Inc.
Inventor: Vladislav Shakhrai , Sergey Demyanov , Mikhail Vasilkovskii , Aleksei Stoliar
CPC classification number: G06T19/20 , G06T7/40 , G06T11/001 , G06T17/20 , G06T2207/20081 , G06T2207/20084 , G06T2210/12 , G06T2210/22 , G06T2219/2016
Abstract: Methods and systems are disclosed for performing operations for generating a photorealistic rendering of an object. The operations include: accessing a set of albedo textures and a machine learning model associated with a real-world object, the set of albedo textures and a machine learning model having been trained based on a plurality of viewpoints of the real-world object; obtaining a three-dimensional (3D) mesh of the real-world object; receiving input that selects a new viewpoint that differs from the plurality of viewpoints of the real-world object; and generating a photorealistic rendering of the real-world object from the new viewpoint based on the 3D mesh of the real-world object, the set of albedo textures, and the machine learning model associated with the real-world object.
-
-
-
-
-
-
-
-
-