公开(公告)号：US20240249523A1

公开(公告)日：2024-07-25

申请号：US18560609

申请日：2022-05-11

Applicant: Google LLC

Inventor： Forrester H. Cole ,  Andrew Zisserman ,  Tali Dekel ,  William Tafel Freeman ,  Erika Lu ,  Michael Rubinstein

IPC: G06V20/40 ,  G06T7/194 ,  G06T7/246 ,  G06T7/73 ,  G06V10/26 ,  G06V10/776 ,  G06V10/82

CPC classification number: G06V20/46 ,  G06T7/194 ,  G06T7/246 ,  G06T7/73 ,  G06V10/26 ,  G06V10/776 ,  G06V10/82 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: The present disclosure provides systems and methods for identifying and extracting object-related effects in videos. Given an ordinary video and a rough segmentation mask overtime of one or more subjects of interest, example systems proposed herein can estimate an omnimatte for each subject—an alpha matte and color image that includes the subject along with all its related time-varying scene elements. Example implementations of the proposed models can be trained only on the input video in a self-supervised manner, without any manual labels, and are generic. For example, the models can produce omnimattes automatically for arbitrary objects and a variety of effects.

公开(公告)号：US12243145B2

公开(公告)日：2025-03-04

申请号：US17927101

申请日：2020-05-22

Applicant: Google LLC

Inventor： Forrester H. Cole ,  Erika Lu ,  Tali Dekel ,  William T. Freeman ,  David Henry Salesin ,  Michael Rubinstein

IPC: G06T13/80 ,  G06V10/44 ,  G06V10/82 ,  G06V20/40 ,  G11B27/00 ,  G11B27/031

Abstract: A computer-implemented method for decomposing videos into multiple layers (212, 213) that can be re-combined with modified relative timings includes obtaining video data including a plurality of image frames (201) depicting one or more objects. For each of the plurality of frames, the computer-implemented method includes generating one or more object maps descriptive of a respective location of at least one object of the one or more objects within the image frame. For each of the plurality of frames, the computer-implemented method includes inputting the image frame and the one or more object maps into a machine-learned layer Tenderer model. (220) For each of the plurality of frames, the computer-implemented method includes receiving, as output from the machine-learned layer Tenderer model, a background layer illustrative of a background of the video data and one or more object layers respectively associated with one of the one or more object maps. The object layers include image data illustrative of the at least one object and one or more trace effects at least partially attributable to the at least one object such that the one or more object layers and the background layer can be re-combined with modified relative timings.

公开(公告)号：US20230206955A1

公开(公告)日：2023-06-29

申请号：US17927101

申请日：2020-05-22

Applicant: Google LLC

Inventor： Forrester H. Cole ,  Erika Lu ,  Tali Dekel ,  William T. Freeman ,  David Henry Salesin ,  Michael Rubinstein

IPC: G11B27/00 ,  G06V10/82 ,  G06V20/40 ,  G11B27/031

CPC classification number: G11B27/005 ,  G06V10/82 ,  G06V20/46 ,  G11B27/031

Abstract: A computer-implemented method for decomposing videos into multiple layers (212, 213) that can be re-combined with modified relative timings includes obtaining video data including a plurality of image frames (201) depicting one or more objects. For each of the plurality of frames, the computer-implemented method includes generating one or more object maps descriptive of a respective location of at least one object of the one or more objects within the image frame. For each of the plurality of frames, the computer-implemented method includes inputting the image frame and the one or more object maps into a machine-learned layer Tenderer model. (220) For each of the plurality of frames, the computer-implemented method includes receiving, as output from the machine-learned layer Tenderer model, a background layer illustrative of a background of the video data and one or more object layers respectively associated with one of the one or more object maps. The object layers include image data illustrative of the at least one object and one or more trace effects at least partially attributable to the at least one object such that the one or more object layers and the background layer can be re-combined with modified relative timings.