公开(公告)号：US20240005537A1

公开(公告)日：2024-01-04

申请号：US18214604

申请日：2023-06-27

Applicant: Apple Inc.

Inventor： Brian Amberg ,  John S. McCarten ,  Nicolas V. Scapel ,  Peter Kaufmann ,  Sebastian Martin

IPC: G06T7/521 ,  G06V40/16 ,  G06T13/40

CPC classification number: G06T7/521 ,  G06V40/176 ,  G06T13/40

Abstract: Various implementations disclosed herein include devices, systems, and methods that generates values for a representation of a face of a user. For example, an example process may include obtaining sensor data (e.g., live data) of a user, wherein the sensor data is associated with a point in time, generating a set of values representing the user based on the sensor data, and providing the set of values, where a depiction of the user at the point in time is displayed based on the set of values. In some implementations, the set of values includes depth values that define three-dimensional (3D) positions of portions of the user relative to multiple 3D positions of points of a projected surface and appearance values (e.g., color, texture, opacity, etc.) that define appearances of the portions of the user.

公开(公告)号：US11856203B1

公开(公告)日：2023-12-26

申请号：US17701498

申请日：2022-03-22

Applicant: Apple Inc.

Inventor： Michael Tschannen ,  Ali Benlalah ,  Anna Volokitin ,  Brian Amberg ,  Sebastian Martin ,  Stefan Brugger

IPC: H04N19/139 ,  H04N19/132 ,  H04N19/42

CPC classification number: H04N19/139 ,  H04N19/132 ,  H04N19/42

Abstract: Advances in deep generative models (DGM) have led to the development of neural face video compression codecs that are capable of using an order of magnitude less data than “traditional” engineered codecs. These “neural” codecs can reconstruct a target image by warping a source image to approximate the content of the target image and using a DGM to compensate for imperfections in the warped source image. The determined warping operation may be encoded and transmitted using less data (e.g., transmitting a small number of keypoints, rather than a dense flow field), leading to the bandwidth savings compared to traditional codecs. However, by relying on a single source image only, these methods can lead to inaccurate reconstructions. The techniques presented herein improve image reconstruction quality while maintaining bandwidth savings, via a combination of using multiple source images (i.e., containing multiple views of the first human subject) and novel feature aggregation techniques.