公开(公告)号：US20200302184A1

公开(公告)日：2020-09-24

申请号：US16823752

申请日：2020-03-19

申请人： Samsung Electronics Co., Ltd.

发明人： Victor Sergeevich LEMPITSKY ,  Aliaksandra Petrovna SHYSHEYA ,  Egor Olegovich ZAKHAROV ,  Egor Andreevich BURKOV

IPC分类号： G06K9/00 ,  G06N3/08

摘要： An electronic device and a controlling method thereof are provided. A controlling method of an electronic device according to the disclosure includes: performing first learning for a neural network model for acquiring a video sequence including a talking head of a random user based on a plurality of learning video sequences including talking heads of a plurality of users, performing second learning for fine-tuning the neural network model based on at least one image including a talking head of a first user different from the plurality of users and first landmark information included in the at least one image, and acquiring a first video sequence including the talking head of the first user based on the at least one image and pre-stored second landmark information using the neural network model for which the first learning and the second learning were performed.

公开(公告)号：US20230169349A1

公开(公告)日：2023-06-01

申请号：US18102161

申请日：2023-01-27

申请人： Samsung Electronics Co., Ltd.

发明人： Victor Sergeevich LEMPITSKY ,  Aliaksandra Petrovna SHYSHEYA ,  Egor Olegovich ZAKHAROV ,  Egor Andreevich BURKOV

IPC分类号： G06N3/088 ,  G06N3/08 ,  G06V20/40 ,  G06V40/16 ,  G06F16/70 ,  G06V40/00 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V40/20

CPC分类号： G06N3/088 ,  G06N3/08 ,  G06V20/46 ,  G06V20/41 ,  G06V40/172 ,  G06F16/70 ,  G06V40/00 ,  G06V40/168 ,  G06V40/169 ,  G06V40/179 ,  G06V10/764 ,  G06V10/7753 ,  G06V10/82 ,  G06V40/20 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30201

摘要： An electronic device and a controlling method thereof are provided. A controlling method of an electronic device according to the disclosure includes: performing first learning for a neural network model for acquiring a video sequence including a talking head of a random user based on a plurality of learning video sequences including talking heads of a plurality of users, performing second learning for fine-tuning the neural network model based on at least one image including a talking head of a first user different from the plurality of users and first landmark information included in the at least one image, and acquiring a first video sequence including the talking head of the first user based on the at least one image and pre-stored second landmark information using the neural network model for which the first learning and the second learning were performed.

公开(公告)号：US20210358197A1

公开(公告)日：2021-11-18

申请号：US17284253

申请日：2019-11-08

申请人： Samsung Electronics Co., Ltd.

发明人： Aliaksandra Petrovna SHYSHEYA ,  Kara-Ali Alibulatovich ALIEV ,  Renat Maratovich BASHIROV ,  Egor Andreevich BURKOV ,  Karim Zhamalovich ISKAKOV ,  Aleksei Aleksandrovich IVAKHNENKO ,  Viktor Sergeevich LEMPITSKY ,  Yury Andreevich MALKOV ,  Igor Igorevich PASECHNIK ,  Dmitry Vladimirovich ULYANOV ,  Alexander Timurovich VAKHITOV ,  Egor Olegovich ZAKHAROV

IPC分类号： G06T15/04 ,  G06T15/20 ,  G06T17/10 ,  G06K9/00 ,  G06N20/00 ,  G06N3/04

摘要： The present invention relates generally to the field of computer vision and computer graphics to produce full body renderings of a person for varying person pose and camera positions and, in particular, to a system and method for synthesizing 2-D image of a person. The method for synthesizing 2-D image of a person comprises: receiving (S101) 3D coordinates of body joint positions of the person defined in a camera coordinate frame, wherein the 3D coordinates of the body joint positions define a pose of the person and a viewpoint of the 2-D image; predicting (S102), using a trained machine learning predictor, a map stack of body part assignments and a map stack of body part coordinates based on the 3D coordinates of the body joint positions, wherein the map stack of body part coordinates defines texture coordinates of pixels of the body parts of the person, the map stack of body part assignments defines weights, each weight indicating a probability of particular pixel to belong to a particular body part of the person; retrieving (S103), from a memory, a map stack of textures for the body parts of the person previously initialized, wherein the map stack of textures comprises values of the pixels of the body parts of the person; and reconstructing (S104) the 2-D image of the person as a weighted combination of the values of the pixels by using the map stack of body part assignments, the map stack of body part coordinates and the map stack of textures. The system for synthesizing 2-D image of a person implements the method for synthesizing 2-D image of a person.