公开(公告)号：US20220261573A1

公开(公告)日：2022-08-18

申请号：US17175441

申请日：2021-02-12

Applicant: Adobe Inc.

Inventor： Jimei YANG ,  Deepali ANEJA ,  Dingzeyu LI ,  Jun SAITO ,  Yang ZHOU

IPC: G06K9/00 ,  H04N21/845 ,  H04N21/8547 ,  G06T7/215 ,  H04N5/06

Abstract: Embodiments are disclosed for re-timing a video sequence to an audio sequence based on the detection of motion beats in the video sequence and audio beats in the audio sequence. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a first input, the first input including a video sequence, detecting motion beats in the video sequence, receiving a second input, the second input including an audio sequence, detecting audio beats in the audio sequence, modifying the video sequence by matching the detected motions beats in the video sequence to the detected audio beats in the audio sequence, and outputting the modified video sequence.

公开(公告)号：US20240005585A1

公开(公告)日：2024-01-04

申请号：US17829120

申请日：2022-05-31

Applicant: Adobe Inc. ,  University of Massachusetts

Inventor： Matthew David FISHER ,  Zhan XU ,  Yang ZHOU ,  Deepali ANEJA ,  Evangelos KALOGERAKIS

IPC: G06T13/80 ,  G06V10/774 ,  G06V10/77 ,  G06V10/762 ,  G06T13/40 ,  G06T7/246

CPC classification number: G06T13/80 ,  G06V10/7747 ,  G06T7/251 ,  G06V10/762 ,  G06T13/40 ,  G06V10/7715

Abstract: Embodiments are disclosed for articulated part extraction using images of animated characters from sprite sheets by a digital design system. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input including a plurality of images depicting an animated character in different poses. The disclosed systems and methods further comprise, for each pair of images in the plurality of images, determining, by a first machine learning model, pixel correspondences between pixels of the pair of images, and determining, by a second machine learning model, pixel clusters representing the animated character, each pixel cluster corresponding to a different structural segment of the animated character. The disclosed systems and methods further comprise selecting a subset of clusters that reconstructs the different poses of the animated character. The disclosed systems and methods further comprise creating a rigged animated character based on the selected subset of clusters.

公开(公告)号：US20240161335A1

公开(公告)日：2024-05-16

申请号：US18055310

申请日：2022-11-14

Applicant: Adobe Inc.

Inventor： Yang ZHOU ,  Jimei YANG ,  Jun SAITO ,  Dingzeyu LI ,  Deepali ANEJA

IPC: G06T7/73 ,  G06F16/683 ,  G06F40/242 ,  G06T7/207

CPC classification number: G06T7/73 ,  G06F16/685 ,  G06F40/242 ,  G06T7/207

Abstract: Embodiments are disclosed for generating a gesture reenactment video sequence corresponding to a target audio sequence using a trained network based on a video motion graph generated from a reference speech video. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a first input including a reference speech video and generating a video motion graph representing the reference speech video, where each node is associated with a frame of the reference video sequence and reference audio features of the reference audio sequence. The disclosed systems and methods further comprise receiving a second input including a target audio sequence, generating target audio features, identifying a node path through the video motion graph based on the target audio features and the reference audio features, and generating an output media sequence based on the identified node path through the video motion graph paired with the target audio sequence.