Photorealistic real-time portrait animation

    公开(公告)号:US12277640B2

    公开(公告)日:2025-04-15

    申请号:US18641472

    申请日:2024-04-22

    Applicant: Snap Inc.

    Abstract: Provided are systems and methods for portrait animation. An example method includes receiving, by a computing device, scenario data including information concerning movements of a first head, receiving, by the computing device, a target image including a second head and a background, determining, by the computing device and based on the target image and the information concerning the movements of the first head, two-dimensional (2D) deformations of the second head in the target image, applying, by the computing device, the 2D deformations to the target image to obtain at least one output frame of an output video, the at least one output frame including the second head displaced according to the movements of the first head, and filling, by the computing device and using a background prediction neural network, a portion of the background in gaps between the displaced second head and the background.

    Message-based processing with assignment of neural network layers to processor clusters

    公开(公告)号:US12277421B2

    公开(公告)日:2025-04-15

    申请号:US18017538

    申请日:2021-07-23

    Applicant: Snap Inc.

    Inventor: Lennart Bamberg

    Abstract: The present application discloses a message based multi-processor system comprising a message exchange network and a plurality of processor clusters to mutually exchange messages. A processor cluster comprises one or more processor cluster elements, and a message generator. The message based multi-processor system is configured as a neural network processor system having a plurality of neural network processing layers each being assigned one or more of the processor clusters. The message generator of a processor cluster comprises a logic module. The logic module, in response to an activation signal of a processor cluster element, selectively generates and transmits a message for each of a set of destination processor clusters in accordance with respective message generation control data for said destination processor clusters stored in a message generator control storage space.

    CUSTOMIZING MODIFIABLE VIDEOS OF MULTIMEDIA MESSAGING APPLICATION

    公开(公告)号:US20250119492A1

    公开(公告)日:2025-04-10

    申请号:US18989143

    申请日:2024-12-20

    Applicant: Snap Inc.

    Abstract: Provided are systems and methods for customizing modifiable videos. An example method includes analyzing recent messages associated with a user in a multimedia messaging application to determine an emotional state of the user, determining, based on the emotional state, a customized property of a modifiable feature of a modifiable video, where the modifiable video includes a preset property of the modifiable feature, replacing the preset property of the modifiable feature with the customized property of the modifiable feature, providing a user interface enabling the user to view the modifiable video and modify the customized property of the modifiable feature to a modified property, and, upon determining that the user has modified the customized property, storing information concerning the emotional state, the customized property, and the modified property to a statistical log.

    AUTOMATIC CAMERA FUNCTION TRIGGERING FOR CONTENT AUGMENTATION

    公开(公告)号:US20250117121A1

    公开(公告)日:2025-04-10

    申请号:US18984594

    申请日:2024-12-17

    Applicant: Snap Inc.

    Abstract: A first content item is shared with a first user by a second user, and is presented on a user device of the first user. A content augmentation was applied to the first content item prior to the sharing thereof with the first user. In response to detecting that the content augmentation was applied to the first content item, a graphical element is presented on the user device together with the first content item. The graphical element is user-selectable to invoke a camera function and to automatically apply the content augmentation to a second content item selected by the first user. In response to detecting user selection of the graphical element, the camera function is triggered on the user device to enable the first user to select a second content item, and the content augmentation is applied to the second content item.

    Combining first user interface content into second user interface

    公开(公告)号:US12271586B2

    公开(公告)日:2025-04-08

    申请号:US18314704

    申请日:2023-05-09

    Applicant: Snap Inc.

    Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for displaying augmented reality content. The program and method provide for displaying, within a messaging application running on a device, a first user interface for viewing a media content item; receiving user input via the first user interface, the user input corresponding to a predefined touch gesture; and in response to receiving the user input, switching, within the messaging application, from the first user interface to a second user interface in which a camera of the device is activated to capture an image, and displaying, via the second user interface, a resized version of the media content item with the image.

    Bending-assisted calibration for extended reality tracking

    公开(公告)号:US12271517B1

    公开(公告)日:2025-04-08

    申请号:US18478352

    申请日:2023-09-29

    Applicant: Snap Inc.

    Abstract: Bending data is used to facilitate tracking operations of an extended reality (XR) device, such as hand tracking or other object tracking operations. The XR device obtains bending data indicative of bending of the XR device to accommodate a body part of a user wearing the XR device. The XR device determines, based on the bending data, whether to use previously identified biometric data in a tracking operation. A mode of the XR device is selected responsive to determining whether to use the previously identified biometric data. The selected mode is used to initialize the tracking operation. The selected mode may be a first mode in which the previously identified biometric data is used in the tracking operation or a second mode in which the previously identified biometric data is not used in the tracking operation.

    LOW-POWER HAND-TRACKING SYSTEM FOR WEARABLE DEVICE

    公开(公告)号:US20250110573A1

    公开(公告)日:2025-04-03

    申请号:US18978713

    申请日:2024-12-12

    Applicant: Snap Inc.

    Abstract: A method for a low-power hand-tracking system is described. In one aspect, a method includes polling a proximity sensor of a wearable device to detect a proximity event, the wearable device includes a low-power processor and a high-power processor, in response to detecting the proximity event, operating a low-power hand-tracking application on the low-power processor based on proximity data from the proximity sensor, and ending an operation of the low-power hand-tracking application in response to at least one of: detecting and recognizing a gesture based on the proximity data, detecting without recognizing the gesture based on the proximity data, or detecting a lack of activity from the proximity sensor within a timeout period based on the proximity data.

    Ultra low power camera pipeline for CV in AR systems

    公开(公告)号:US12267585B2

    公开(公告)日:2025-04-01

    申请号:US17485761

    申请日:2021-09-27

    Applicant: Snap Inc.

    Abstract: An eyewear device having an image processor operable in a camera pipeline for computer vision (CV) and in augmented reality (AR) systems. The image processor is configured to selectively control a plurality of cameras to provide images having a first resolution in the high power AR mode, and to provide the images having a second resolution in the low power CV mode. The first resolution is higher than the second resolution, and the plurality of cameras consume less power in the CV mode than the AR mode. The image processor controls the camera pipeline to process the first resolution high IQ images from the plurality of cameras to operate in the AR mode, and controls the camera pipeline to process the second resolution lower IQ images from the plurality of cameras to operate in the CV mode. Substantial power is saved by reducing the resolution of the images using downscaling in the cameras themselves in the CV mode.

Patent Agency Ranking