公开(公告)号：US20240233229A1

公开(公告)日：2024-07-11

申请号：US18007867

申请日：2021-11-08

Applicant: NVIDIA Corporation

Inventor： Evgeny Aleksandrovich Tumanov ,  Dmitry Aleksandrovich Korobchenko ,  Simon Yuen ,  Kevin Margo

IPC: G06T13/20 ,  G06T13/40

CPC classification number: G06T13/205 ,  G06T13/40

Abstract: In various examples, animations may be generated using audio-driven body animation synthesized with voice tempo. For example, full body animation may be driven from an audio input representative of recorded speech, where voice tempo (e.g., a number of phonemes per unit time) may be used to generate a 1D audio signal for comparing to datasets including data samples that each include an animation and a corresponding 1D audio signal. One or more loss functions may be used to compare the 1D audio signal from the input audio to the audio signals of the datasets, as well as to compare joint information of joints of an actor between animations of two or more data samples, in order to identify optimal transition points between the animations. The animations may then be stitched together—e.g., using interpolation and/or a neural network trained to seamlessly stitch sequences together—using the transition points.