-
公开(公告)号:US20250142182A1
公开(公告)日:2025-05-01
申请号:US18584210
申请日:2024-02-22
Applicant: ADOBE INC.
Inventor: Joanna Irena Materzynska , Richard Zhang , Elya Shechtman , Josef Sivic , Bryan Christopher Russell
IPC: H04N21/81 , H04N21/488
Abstract: Systems and methods include generating synthetic videos based on a custom motion. A video generation system obtains a text prompt including an object and a custom motion token. The custom motion token represents a custom motion. The system encodes the text prompt to obtain a text embedding. Subsequently, a video generation model generates a synthetic video depicting the object performing the custom motion based on the text embedding using a video generation model.
-
公开(公告)号:US12079948B2
公开(公告)日:2024-09-03
申请号:US17942101
申请日:2022-09-09
Applicant: ADOBE INC.
Inventor: Taesung Park , Richard Zhang , Elya Shechtman
IPC: G06T19/20 , G06F3/04847 , G06F40/284 , G06F40/289 , G06V10/774 , G06V10/776 , G06V10/82
CPC classification number: G06T19/20 , G06F3/04847 , G06F40/289 , G06V10/774 , G06V10/776 , G06V10/82 , G06F40/284 , G06T2200/24 , G06T2210/61
Abstract: Various disclosed embodiments are directed to changing parameters of an input image or multidimensional representation of the input image based on a user request to change such parameters. An input image is first received. A multidimensional image that represents the input image in multiple dimensions is generated via a model. A request to change at least a first parameter to a second parameter is received via user input at a user device. Such request is a request to edit or generate the multidimensional image in some way. For instance, the request may be to change the light source position or camera position from a first set of coordinates to a second set of coordinates.
-
公开(公告)号:US20230360376A1
公开(公告)日:2023-11-09
申请号:US17744995
申请日:2022-05-16
Applicant: Adobe Inc.
Inventor: Tobias Hinz , Taesung Park , Richard Zhang , Matthew David Fisher , Difan Liu , Evangelos Kalogerakis
IPC: G06V10/774 , G06V10/22 , G06T3/40
CPC classification number: G06V10/7753 , G06V10/235 , G06T3/4046
Abstract: Semantic fill techniques are described that support generating fill and editing images from semantic inputs. A user input, for example, is received by a semantic fill system that indicates a selection of a first region of a digital image and a corresponding semantic label. The user input is utilized by the semantic fill system to generate a guidance attention map of the digital image. The semantic fill system leverages the guidance attention map to generate a sparse attention map of a second region of the digital image. A semantic fill of pixels is generated for the first region based on the semantic label and the sparse attention map. The edited digital image is displayed in a user interface.
-
公开(公告)号:US11762951B2
公开(公告)日:2023-09-19
申请号:US16951782
申请日:2020-11-18
Applicant: Adobe Inc.
Inventor: Elya Shechtman , William Peebles , Richard Zhang , Jun-Yan Zhu , Alyosha Efros
IPC: G06F18/21 , G06N3/08 , G06T3/00 , G06F18/214 , G06N3/045
CPC classification number: G06F18/217 , G06F18/214 , G06N3/045 , G06N3/08 , G06T3/0068
Abstract: Embodiments are disclosed for generative image congealing which provides an unsupervised learning technique that learns transformations of real data to improve the image quality of GANs trained using that image data. In particular, in one or more embodiments, the disclosed systems and methods comprise generating, by a spatial transformer network, an aligned real image for a real image from an unaligned real dataset, providing, by the spatial transformer network, the aligned real image to an adversarial discrimination network to determine if the aligned real image resembles aligned synthetic images generated by a generator network, and training, by a training manager, the spatial transformer network to learn updated transformations based on the determination of the adversarial discrimination network.
-
公开(公告)号:US20230102055A1
公开(公告)日:2023-03-30
申请号:US18058163
申请日:2022-11-22
Applicant: Adobe Inc.
Inventor: Taesung Park , Richard Zhang , Oliver Wang , Junyan Zhu , Jingwan Lu , Elya Shechtman , Alexei A. Efros
Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a modified digital image from extracted spatial and global codes. For example, the disclosed systems can utilize a global and spatial autoencoder to extract spatial codes and global codes from digital images. The disclosed systems can further utilize the global and spatial autoencoder to generate a modified digital image by combining extracted spatial and global codes in various ways for various applications such as style swapping, style blending, and attribute editing.
-
6.
公开(公告)号:US11468294B2
公开(公告)日:2022-10-11
申请号:US16798271
申请日:2020-02-21
Applicant: Adobe Inc.
Inventor: Richard Zhang , Sylvain Philippe Paris , Junyan Zhu , Aaron Phillip Hertzmann , Jacob Minyoung Huh
Abstract: A target image is projected into a latent space of generative model by determining a latent vector by applying a gradient-free technique and a class vector by applying a gradient-based technique. An image is generated from the latent and class vectors, and a loss function is used to determine a loss between the target image and the generated image. This determining of the latent vector and the class vector, generating an image, and using the loss function is repeated until a loss condition is satisfied. In response to the loss condition being satisfied, the latent and class vectors that resulted in the loss condition being satisfied are identified as the final latent and class vectors, respectively. The final latent and class vectors are provided to the generative model and multiple weights of the generative model are adjusted to fine-tune the generative model.
-
公开(公告)号:US20220122305A1
公开(公告)日:2022-04-21
申请号:US17384273
申请日:2021-07-23
Applicant: Adobe Inc.
Inventor: Cameron Smith , Ratheesh Kalarot , Wei-An Lin , Richard Zhang , Niloy Mitra , Elya Shechtman , Shabnam Ghadar , Zhixin Shu , Yannick Hold-Geoffrey , Nathan Carr , Jingwan Lu , Oliver Wang , Jun-Yan Zhu
Abstract: An improved system architecture uses a pipeline including an encoder and a Generative Adversarial Network (GAN) including a generator neural network to generate edited images with improved speed, realism, and identity preservation. The encoder produces an initial latent space representation of an input image by encoding the input image. The generator neural network generates an initial output image by processing the initial latent space representation of the input image. The system generates an optimized latent space representation of the input image using a loss minimization technique that minimizes a loss between the input image and the initial output image. The loss is based on target perceptual features extracted from the input image and initial perceptual features extracted from the initial output image. The system outputs the optimized latent space representation of the input image for downstream use.
-
公开(公告)号:US20220122222A1
公开(公告)日:2022-04-21
申请号:US17384283
申请日:2021-07-23
Applicant: Adobe Inc.
Inventor: Cameron Smith , Ratheesh Kalarot , Wei-An Lin , Richard Zhang , Niloy Mitra , Elya Shechtman , Shabnam Ghadar , Zhixin Shu , Yannick Hold-Geoffrey , Nathan Carr , Jingwan Lu , Oliver Wang , Jun-Yan Zhu
Abstract: An improved system architecture uses a Generative Adversarial Network (GAN) including a specialized generator neural network to generate multiple resolution output images. The system produces a latent space representation of an input image. The system generates a first output image at a first resolution by providing the latent space representation of the input image as input to a generator neural network comprising an input layer, an output layer, and a plurality of intermediate layers and taking the first output image from an intermediate layer, of the plurality of intermediate layers of the generator neural network. The system generates a second output image at a second resolution different from the first resolution by providing the latent space representation of the input image as input to the generator neural network and taking the second output image from the output layer of the generator neural network.
-
公开(公告)号:US20220121932A1
公开(公告)日:2022-04-21
申请号:US17384378
申请日:2021-07-23
Applicant: Adobe Inc.
Inventor: Ratheesh Kalarot , Wei-An Lin , Cameron Smith , Zhixin Shu , Baldo Faieta , Shabnam Ghadar , Jingwan Lu , Aliakbar Darabi , Jun-Yan Zhu , Niloy Mitra , Richard Zhang , Elya Shechtman
Abstract: Systems and methods train an encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The encoder is trained by providing an input training image to the encoder and producing, by the encoder, a latent space representation of the input training image. The latent space representation is provided as input to the GAN to generate a generated training image. A latent code is sampled from a latent space associated with the GAN and the sampled latent code is provided as input to the GAN. The GAN generates a synthetic training image based on the sampled latent code. The sampled latent code is provided as input to the encoder to produce a synthetic training code. The encoder is updated by minimizing a loss between the generated training image and the input training image, and the synthetic training code and the sampled latent code.
-
公开(公告)号:US11232607B2
公开(公告)日:2022-01-25
申请号:US16751959
申请日:2020-01-24
Applicant: Adobe Inc.
Inventor: Nishant Kumar , Vikas Sharma , Shantanu Agarwal , Sameer Bhatt , Rupali Arora , Richard Zhang , Anuradha Yadav , Jingwan Lu , Matthew David Fisher
Abstract: In implementations of adding color to digital images, an image colorization system can display a digital image to be color adjusted in an image editing interface and convert pixel content of the digital image to a LAB color space. The image colorization system can determine a lightness value (L) in the LAB color space of the pixel content of the digital image at a specified point on the digital image, and determine colors representable in an RGB color space based on combinations of A,B value pairs with the lightness value (L) in the LAB color space. The image colorization system can then determine a range of the colors for display in a color gamut in the image editing interface, the range of the colors corresponding to the A,B value pairs with the lightness value (L) of the pixel content at the specified point on the digital image.
-
-
-
-
-
-
-
-
-