Real-time motion and appearance transfer

    公开(公告)号:US12198398B2

    公开(公告)日:2025-01-14

    申请号:US17645366

    申请日:2021-12-21

    Applicant: Snap Inc.

    Abstract: Methods and systems are disclosed for performing operations for transferring motion from one real-world object to another in real-time. The operations comprise receiving a first video that includes a depiction of a first real-world object and extracting an appearance of the first real-world object from the video. The operations comprise obtaining a second video that includes a depiction of a second real-world object and extracting motion of the second real-world object from the second video. The operations comprise applying the motion of the second real-world object extracted from the second video to the appearance of the first real-world object extracted from the first video. The operations comprise generating a third video that includes a depiction of the first real-world object having the appearance of the first real-world object and the motion of the second real-world object.

    Background replacement using neural radiance field

    公开(公告)号:US12177600B2

    公开(公告)日:2024-12-24

    申请号:US17852919

    申请日:2022-06-29

    Applicant: Snap Inc.

    Abstract: Aspects of the present disclosure involve a system for providing virtual experiences. The system accesses an image depicting a person and one or more camera parameters representing a viewpoint associated with a camera used to capture the image. The system extracts a portion of the image comprising the depiction of the person. The system processes, by a neural radiance field (NeRF) machine learning model, the one or more camera parameters to render an estimated depiction of a scene from the viewpoint associated with the camera used to capture the image. The system combines the portion of the image comprising the depiction of the person with the estimated depiction of the scene to generate an output image and causes the output image to be presented on a client device.

    Interactively defining an object segmentation

    公开(公告)号:US12159412B2

    公开(公告)日:2024-12-03

    申请号:US17650918

    申请日:2022-02-14

    Applicant: Snap Inc.

    Abstract: Methods and systems are disclosed for performing operations for segmenting an object. The operations include receiving an image that includes a depiction of a first object; receiving a first segmentation dataset; applying a first machine learning technique to the first segmentation dataset and the image to estimate a first segmentation of the first object depicted in the image; generating a second segmentation dataset that includes the estimated first segmentation and a correction to the estimated first segmentation of the first object; applying the first machine learning technique to the second segmentation dataset and the image to estimate a second segmentation of the first object depicted in the image; and applying an augmented reality experience to the image based on the estimated second segmentation of the first object.

    AUGMENTED REALITY EXPERIENCE WITH LIGHTING ADJUSTMENT

    公开(公告)号:US20240282015A1

    公开(公告)日:2024-08-22

    申请号:US18111632

    申请日:2023-02-20

    Applicant: Snap Inc.

    Abstract: Aspects of the present disclosure involve a system for an augmented reality (AR) try-on experience with lighting adjustment. The system accesses an image that depicts a real-world person. The system retrieves an AR fashion item. The system applies the image and the AR fashion item to a machine learning model to estimate a lighting adjustment for the AR fashion item based on lighting properties of the real-world person depicted in the image, the machine learning model trained to match lighting properties of images depicting real-world objects to lighting properties of AR objects and vice versa. The system combines the AR fashion item with the image that depicts the real-world person based on the estimated lighting adjustment to generate a modified image.

    Applying pregenerated virtual experiences in new location

    公开(公告)号:US12020386B2

    公开(公告)日:2024-06-25

    申请号:US17848087

    申请日:2022-06-23

    Applicant: Snap Inc.

    CPC classification number: G06T19/006 G06N3/02 G06T19/20

    Abstract: Aspects of the present disclosure involve a system for providing virtual experiences. The system performs operations including selecting, by a messaging application, a virtual experience that represents a previously captured real-world environment at a first location; accessing an image representing a new real-world environment at a second location, the image depicting a plurality of real-world objects; receiving input that selects a first real-world object from the plurality of real-world objects depicted in the image; and modifying the image, accessed at the second location, based on the virtual experience to depict the previously captured real-world environment with the first real-world object.

    VIRTUAL CLOTHING TRY-ON
    37.
    发明公开

    公开(公告)号:US20240062444A1

    公开(公告)日:2024-02-22

    申请号:US18499166

    申请日:2023-10-31

    Applicant: Snap Inc.

    Abstract: A messaging system performs virtual clothing try-on. A method of virtual clothing try-on may include accessing a target garment image and a person image of a person wearing a source garment and processing the person image to generate a source garment mask and a person mask. The method may further include processing the source garment mask, the person mask, the target garment image, and a target garment mask to generate a warping, the warping indicating a warping to apply to the target garment image. The method may further include processing the target garment to warp the target garment in accordance with the warping to generate a warped target garment image, processing the warped target garment image to blend with the person image to generate a person with a blended target garment image, and processing the person with blended target garment image to fill in holes to generate an output image.

    Real-time upper-body garment exchange

    公开(公告)号:US11880947B2

    公开(公告)日:2024-01-23

    申请号:US17557277

    申请日:2021-12-21

    Applicant: Snap Inc.

    CPC classification number: G06T19/006 G06T7/215 G06T7/70 G06T19/20

    Abstract: Methods and systems are disclosed for performing operations for transferring garments from one real-world object to another in real time. The operations comprise receiving a first video that includes a depiction of a first person wearing a first upper-body garment in a first pose and obtaining a second video that includes a depiction of a second person wearing a second upper-body garment in a second pose. A pose of the second person depicted in the second video is modified to match the first pose of the first person depicted in the first video. The operations comprise generating an upper-body segmentation of the second upper-body garment which the second person is wearing in the second video in the modified pose and replacing the first upper-body garment worn by the first person in the first video with the second upper-body garment based on the upper-body segmentation.

    AUGMENTED REALITY ANTHROPOMORPHIZATION SYSTEM

    公开(公告)号:US20230290026A1

    公开(公告)日:2023-09-14

    申请号:US18320628

    申请日:2023-05-19

    Applicant: Snap Inc.

    CPC classification number: G06T11/60 G06T13/80 G06V20/10

    Abstract: Embodiments of the present disclosure relate generally to systems for enhancing a first media item through the addition of a supplemental second media item. A user may provide a request to enhance a selected media item, and in response, an enhancement system retrieves and presents a curated collection of supplemental content to be added to the media, to the user. The user may review the curated collection of supplemental content, for example by providing a tactile input to scroll through the curated collection of content.

Patent Agency Ranking