-
公开(公告)号：US10896535B2
公开(公告)日：2021-01-19
申请号：US16430204
申请日：2019-06-03
申请人： Pinscreen, Inc.
发明人： Hao Li , Koki Nagano , Jaewoo Seo , Lingyu Wei , Jens Fursund
摘要： A system and method for generating real-time facial animation is disclosed. The system relies upon pre-generating a series of key expression images from a single neutral image using a pre-trained generative adversarial neural network. The key expression images are used to generate a set of FACS expressions and associated textures which may be applied to a three-dimensional model to generate facial animation. The FACS expressions and textures may be provided to a mobile device to enable that mobile device to generate convincing three-dimensional avatars in real-time with convincing animation in a processor non-intensive way through a blending process using the pre-determined FACS expressions and textures.
-
公开(公告)号：US10535163B2
公开(公告)日：2020-01-14
申请号：US16119907
申请日：2018-08-31
申请人： Pinscreen, Inc.
发明人： Hao Li , Liwen Hu , Lingyu Wei , Koki Nagano , Jaewoo Seo , Jens Fursund , Shunsuke Saito
摘要： A system for generating three-dimensional facial models including photorealistic hair and facial textures includes creating a facial model with reliance upon neural networks based upon a single two-dimensional input image. The photorealistic hair is created by finding a subset of similar three-dimensional polystrip hairstyles from a large database of polystrip hairstyles, selecting the most-alike polystrip hairstyle, deforming that polystrip hairstyle to better fit the hair of the two-dimensional image. Then, collisions and bald spots are corrected, and suitable textures are applied. Finally, the facial model and polystrip hairstyle are combined into a final three-dimensional avatar.
-
公开(公告)号：US10198624B2
公开(公告)日：2019-02-05
申请号：US15438551
申请日：2017-02-21
申请人： Pinscreen, Inc.
发明人： Hao Li , Tianye Li , Shunsuke Saito
摘要： There is disclosed a system and method of performing facial recognition from RGB image data. The method includes generating a lower-resolution image from the RGB image data, performing a convolution of the lower-resolution image data to derive a probability map identifying probable facial regions and a probable non-facial regions, and performing a first deconvolution on the lower-resolution image using a bilinear interpolation layer to derive a set of coarse facial segments. The method further includes performing a second deconvolution on the lower-resolution image using a series of unpooling, deconvolution, and rectification layers to derive a set of fine facial segments, concatenating the set of coarse facial segments to the set of fine facial segments to create an image matrix made up of a set of facial segments, and generating a binary facial mask identifying probable facial regions and probable non-facial regions from the image matrix.
-
公开(公告)号：US12033277B2
公开(公告)日：2024-07-09
申请号：US17573159
申请日：2022-01-11
申请人： Pinscreen, Inc.
发明人： Huiwen Luo , Koki Nagano , Zejian Wang , Lingyu Wei , Liwen Hu , Hao Li
CPC分类号： G06T17/00 , G06N3/045 , G06T15/506 , G06V40/168
摘要： A system, method, and apparatus for generating a normalized three-dimensional model of a human face from a single unconstrained two-dimensional image of the human face. The system includes a processor that executes instructions including receiving the single unconstrained two-dimensional image of the human face, using an inference network to determine an inferred normalized three-dimensional model of the human face based on the single unconstrained two-dimensional image of the human face, and using a refinement network to iteratively determine the normalized three-dimensional model of the human face with a neutral expression and unshaded albedo textures under diffuse lighting conditions based on the inferred normalized three-dimensional model of the human face.
-
公开(公告)号：US20170243053A1
公开(公告)日：2017-08-24
申请号：US15438551
申请日：2017-02-21
申请人： Pinscreen, Inc.
发明人： Hao Li , Tianye Li , Shunsuke Saito
CPC分类号： G06K9/00281 , G06K9/00234 , G06K9/4628 , G06T7/11 , G06T7/143 , G06T7/194 , G06T2207/10024 , G06T2207/20016 , G06T2207/20076 , G06T2207/20081 , G06T2207/30201
摘要： There is disclosed a system and method of performing facial recognition from RGB image data. The method includes generating a lower-resolution image from the RGB image data, performing a convolution of the lower-resolution image data to derive a probability map identifying probable facial regions and a probable non-facial regions, and performing a first deconvolution on the lower-resolution image using a bilinear interpolation layer to derive a set of coarse facial segments. The method further includes performing a second deconvolution on the lower-resolution image using a series of unpooling, deconvolution, and rectification layers to derive a set of fine facial segments, concatenating the set of coarse facial segments to the set of fine facial segments to create an image matrix made up of a set of facial segments, and generating a binary facial mask identifying probable facial regions and probable non-facial regions from the image matrix.
-
公开(公告)号：US10217261B2
公开(公告)日：2019-02-26
申请号：US15438546
申请日：2017-02-21
申请人： Pinscreen, Inc.
发明人： Hao Li , Joseph J. Lim , Kyle Olszewski
摘要： There is disclosed a system and method for training a set of expression and neutral convolutional neural networks using a single performance mapped to a set of known phonemes and visemes in the form predetermined sentences and facial expressions. Then, subsequent training of the convolutional neural networks can occur using temporal data derived from audio data within the original performance mapped to a set of professionally-created three dimensional animations. Thereafter, with sufficient training, the expression and neutral convolutional neural networks can generate facial animations from facial image data in real-time without individual specific training.
-
公开(公告)号：US20180374242A1
公开(公告)日：2018-12-27
申请号：US16119907
申请日：2018-08-31
申请人： Pinscreen, Inc.
发明人： Hao Li , Liwen Hu , Lingyu Wei , Koki Nagano , Jaewoo Seo , Jens Fursund
摘要： A system for generating three-dimensional facial models including photorealistic hair and facial textures includes creating a facial model with reliance upon neural networks based upon a single two-dimensional input image. The photorealistic hair is created by finding a subset of similar three-dimensional polystrip hairstyles from a large database of polystrip hairstyles, selecting the most-alike polystrip hairstyle, deforming that polystrip hairstyle to better fit the hair of the two-dimensional image. Then, collisions and bald spots are corrected, and suitable textures are applied. Finally, the facial model and polystrip hairstyle are combined into a final three-dimensional avatar.
-
公开(公告)号：US20180158240A1
公开(公告)日：2018-06-07
申请号：US15829064
申请日：2017-12-01
申请人： Pinscreen, Inc.
发明人： Shunsuke Saito , Cosimo Wei , Liwen Hu , Hao Li
CPC分类号： G06T17/20 , G06K9/00214 , G06K9/00228 , G06K9/00241 , G06K9/66 , G06T11/001 , G06T15/04
摘要： A method for generating three-dimensional facial models and photorealistic textures from inferences using deep neural networks relies upon generating a low frequency and a high frequency albedo map of the full and partial face, respectively. Then, the high frequency albedo map may be used for comparison with correlation matrices generated by a neural network trained by a large scale, high-resolution facial dataset with simulated partial visibility. The corresponding correlation matrices of the complete facial textures can then be retrieved. Finally, a full facial texture map may be synthesized, using convex combinations of the correlation matrices. A photorealistic facial texture for the three-dimensional face rendering can be obtained through optimization using the deep neural network and a loss function that incorporates the blended target correlation matrices.
-
公开(公告)号：US11908233B2
公开(公告)日：2024-02-20
申请号：US17343575
申请日：2021-06-09
申请人： Pinscreen, Inc.
发明人： Koki Nagano , Huiwen Luo , Zejian Wang , Jaewoo Seo , Liwen Hu , Lingyu Wei , Hao Li
CPC分类号： G06V40/161 , G06T5/006 , G06T2207/20081 , G06T2207/30201
摘要： A system, method, and apparatus for generating a normalization of a single two-dimensional image of an unconstrained human face. The system receives the single two-dimensional image of the unconstrained human face, generates an undistorted face based on the unconstrained human face by removing perspective distortion from the unconstrained human face via a perspective undistortion network, generates an evenly lit face based on the undistorted face by normalizing lighting of the undistorted face via a lighting translation network, and generates a frontalized and neutralized expression face based on the evenly lit face via an expression neutralization network.
-
公开(公告)号：US20220222892A1
公开(公告)日：2022-07-14
申请号：US17573159
申请日：2022-01-11
申请人： Pinscreen, Inc.
发明人： Huiwen Luo , Koki Nagano , Zejian Wang , Lingyu Wei , Liwen Hu , Hao Li
摘要： A system, method, and apparatus for generating a normalized three-dimensional model of a human face from a single unconstrained two-dimensional image of the human face. The system includes a processor that executes instructions including receiving the single unconstrained two-dimensional image of the human face, using an inference network to determine an inferred normalized three-dimensional model of the human face based on the single unconstrained two-dimensional image of the human face, and using a refinement network to iteratively determine the normalized three-dimensional model of the human face with a neutral expression and unshaded albedo textures under diffuse lighting conditions based on the inferred normalized three-dimensional model of the human face.
-
-
-
-
-
-
-
-
-
搜索结果
13 条记录 (耗时 0.013 秒)
