公开(公告)号：US20200273180A1

公开(公告)日：2020-08-27

申请号：US16761582

申请日：2018-11-13

Applicant: APPLE INC.

Inventor： Peter KAUFMANN ,  Daniel KURZ ,  Brian AMBERG ,  Yanghai TSIN

IPC: G06T7/246 ,  G06T7/73 ,  G06T7/292 ,  G06T17/00 ,  G06T7/55

Abstract: Various implementations disclosed herein include devices, systems, and methods that use event camera data to track deformable objects such as faces, hands, and other body parts. One exemplary implementation involves receiving a stream of pixel events output by an event camera. The device tracks the deformable object using this data. Various implementations do so by generating a dynamic representation of the object and modifying the dynamic representation of the object in response to obtaining additional pixel events output by the event camera. In some implementations, generating the dynamic representation of the object involves identifying features disposed on the deformable surface of the object using the stream of pixel events. The features are determined by identifying patterns of pixel events. As new event stream data is received, the patterns of pixel events are recognized in the new data and used to modify the dynamic representation of the object.

公开(公告)号：US20190251728A1

公开(公告)日：2019-08-15

申请号：US16276535

申请日：2019-02-14

Applicant: Apple Inc.

Inventor： Justin D. STOYLES ,  Alexandre R. MOHA ,  Nicolas V. SCAPEL ,  Guillaume P. BARLIER ,  Aurelio GUZMAN ,  Bruno M. SOMMER ,  Nina DAMASKY ,  Thibaut WEISE ,  Thomas GOOSSENS ,  Hoan PHAM ,  Brian AMBERG

IPC: G06T13/40 ,  G06K9/00 ,  H04N7/14 ,  G11B27/036 ,  G06T17/20 ,  G06T13/20 ,  G11B27/10

CPC classification number: G06T13/40 ,  G06K9/00228 ,  G06K9/00302 ,  G06T13/205 ,  G06T17/205 ,  G11B27/036 ,  G11B27/10 ,  H04N7/147 ,  H04N2007/145

Abstract: Systems and methods for generating a video of an emoji that has been puppeted using inputs from image, depth, and audio. The inputs can capture facial expressions of a user, eye, eyebrow, mouth, and head movements. A pose, held by the user, can be detected that can be used to generate supplemental animation. The emoji can further be animated using physical properties associated with the emoji and captured movements. An emoji of a dog can have its ears move in response to an up-and-down movement, or a shaking of the head. The video can be sent in a message to one or more recipients. A sending device can render the puppeted video in accordance with hardware and software capabilities of a recipient's computer device.