-
11.发明申请公开(公告)号：US20200372284A1
公开(公告)日：2020-11-26
申请号：US16616235
申请日：2019-10-16
Applicant: Google LLC
Inventor： Christoph Rhemann , Abhimitra Meka , Matthew Whalen , Jessica Lynn Busch , Sofien Bouaziz , Geoffrey Douglas Harvey , Andrea Tagliasacchi , Jonathan Taylor , Paul Debevec , Peter Joseph Denny , Sean Ryan Francesco Fanello , Graham Fyffe , Jason Angelo Dourgarian , Xueming Yu , Adarsh Prakash Murthy Kowdle , Julien Pascal Christophe Valentin , Peter Christopher Lincoln , Rohit Kumar Pandey , Christian Häne , Shahram Izadi
Abstract: Methods, systems, and media for relighting images using predicted deep reflectance fields are provided. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample includes (i) a group of one-light-at-a-time (OLAT) images that have each been captured when one light of a plurality of lights arranged on a lighting structure has been activated, (ii) a group of spherical color gradient images that have each been captured when the plurality of lights arranged on the lighting structure have been activated to each emit a particular color, and (iii) a lighting direction, wherein each image in the group of OLAT images and each of the spherical color gradient images are an image of a subject, and wherein the lighting direction indicates a relative orientation of a light to the subject; training a convolutional neural network using the group of training samples, wherein training the convolutional neural network comprises: for each training iteration in a series of training iterations and for each training sample in the group of training samples: generating an output predicted image, wherein the output predicted image is a representation of the subject associated with the training sample with lighting from the lighting direction associated with the training sample; identifying a ground-truth OLAT image included in the group of OLAT images for the training sample that corresponds to the lighting direction for the training sample; calculating a loss that indicates a perceptual difference between the output predicted image and the identified ground-truth OLAT image; and updating parameters of the convolutional neural network based on the calculated loss; identifying a test sample that includes a second group of spherical color gradient images and a second lighting direction; and generating a relit image of the subject included in each of the second group of spherical color gradient images with lighting from the second lighting direction using the trained convolutional neural network.
-
公开(公告)号：US20240212106A1
公开(公告)日：2024-06-27
申请号：US18554960
申请日：2021-04-28
Applicant: Google LLC
Inventor： Chloe LeGendre , Paul Debevec , Sean Ryan Francesco Fanello , Rohit Kumar Pandey , Sergio Orts Escolano , Christian Haene , Sofien Bouaziz
CPC classification number: G06T5/50 , G06T7/194 , G06V10/56 , G06V10/60 , H04N5/272 , G06T2207/20221
Abstract: Apparatus and methods related to applying lighting models to images are provided. An example method includes receiving, via a computing device, an image comprising a subject. The method further includes relighting, via a neural network, a foreground of the image to maintain a consistent lighting of the foreground with a target illumination. The relighting is based on a per-pixel light representation indicative of a surface geometry of the foreground. The light representation includes a specular component, and a diffuse component, of surface reflection. The method additionally includes predicting, via the neural network, an output image comprising the subject in the relit foreground. One or more neural networks can be trained to perform one or more of the aforementioned aspects.
-
13.发明授权公开(公告)号：US10997457B2
公开(公告)日：2021-05-04
申请号：US16616235
申请日：2019-10-16
Applicant: Google LLC
Inventor： Christoph Rhemann , Abhimitra Meka , Matthew Whalen , Jessica Lynn Busch , Sofien Bouaziz , Geoffrey Douglas Harvey , Andrea Tagliasacchi , Jonathan Taylor , Paul Debevec , Peter Joseph Denny , Sean Ryan Francesco Fanello , Graham Fyffe , Jason Angelo Dourgarian , Xueming Yu , Adarsh Prakash Murthy Kowdle , Julien Pascal Christophe Valentin , Peter Christopher Lincoln , Rohit Kumar Pandey , Christian Häne , Shahram Izadi
Abstract: Methods, systems, and media for relighting images using predicted deep reflectance fields are provided. In some embodiments, the method comprises: identifying a group of training samples, wherein each training sample includes (i) a group of one-light-at-a-time (OLAT) images that have each been captured when one light of a plurality of lights arranged on a lighting structure has been activated, (ii) a group of spherical color gradient images that have each been captured when the plurality of lights arranged on the lighting structure have been activated to each emit a particular color, and (iii) a lighting direction, wherein each image in the group of OLAT images and each of the spherical color gradient images are an image of a subject, and wherein the lighting direction indicates a relative orientation of a light to the subject; training a convolutional neural network using the group of training samples, wherein training the convolutional neural network comprises: for each training iteration in a series of training iterations and for each training sample in the group of training samples: generating an output predicted image, wherein the output predicted image is a representation of the subject associated with the training sample with lighting from the lighting direction associated with the training sample; identifying a ground-truth OLAT image included in the group of OLAT images for the training sample that corresponds to the lighting direction for the training sample; calculating a loss that indicates a perceptual difference between the output predicted image and the identified ground-truth OLAT image; and updating parameters of the convolutional neural network based on the calculated loss; identifying a test sample that includes a second group of spherical color gradient images and a second lighting direction; and generating a relit image of the subject included in each of the second group of spherical color gradient images with lighting from the second lighting direction using the trained convolutional neural network.
-
公开(公告)号：US20190102936A1
公开(公告)日：2019-04-04
申请号：US16151276
申请日：2018-10-03
Applicant: GOOGLE LLC
Inventor： Ivan Neulander , Chloe LeGendre , Paul Debevec
IPC: G06T15/80 , G06T7/70 , G06T7/40 , G06T3/40 , G06T7/11 , G06T15/04 , G06T19/00 , G02B27/01 , G06T15/00
Abstract: Systems and methods for lighting inserted content are provided. For example, the inserted content may include augmented reality content that is inserted into an image of a physical space. An example system and method may include determining a location within an image to insert content. For example, the image may be captured by a camera device. The example system and method may also include identifying a region of the image based on the determined location to insert the content, determining at least one lighting parameter based on the identified region, and rendering the content using the determined at least one lighting parameter.
-
-
-
Search Results
14
records
(took 0.013 seconds)
