-
公开(公告)号:US20220148242A1
公开(公告)日:2022-05-12
申请号:US17091440
申请日:2020-11-06
Applicant: Adobe Inc.
Inventor: Bryan Russell , Taesung Park , Richard Zhang , Junyan Zhu , Alexander Andonian
Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize a contrastive perceptual loss to modify neural networks for generating synthetic digital content items. For example, the disclosed systems generate a synthetic digital content item based on a guide input to a generative neural network. The disclosed systems utilize an encoder neural network to generate encoded representations of the synthetic digital content item and a corresponding ground-truth digital content item. Additionally, the disclosed systems sample patches from the encoded representations of the encoded digital content items and then determine a contrastive loss based on the perceptual distances between the patches in the encoded representations. Furthermore, the disclosed systems jointly update the parameters of the generative neural network and the encoder neural network utilizing the contrastive loss.
-
公开(公告)号:US20220122306A1
公开(公告)日:2022-04-21
申请号:US17468487
申请日:2021-09-07
Applicant: Adobe Inc.
Inventor: Wei-An Lin , Baldo Faieta , Cameron Smith , Elya Shechtman , Jingwan Lu , Jun-Yan Zhu , Niloy Mitra , Ratheesh Kalarot , Richard Zhang , Shabnam Ghadar , Zhixin Shu
IPC: G06T11/60 , G06F3/0484 , G06N3/08 , G06N3/04
Abstract: Systems and methods dynamically adjust an available range for editing an attribute in an image. An image editing system computes a metric for an attribute in an input image as a function of a latent space representation of the input image and a filtering vector for editing the input image. The image editing system compares the metric to a threshold. If the metric exceeds the threshold, then the image editing system selects a first range for editing the attribute in the input image. If the metric does not exceed the threshold, a second range is selected. The image editing system causes display of a user interface for editing the input image comprising an interface element for editing the attribute within the selected range.
-
公开(公告)号:US20220122221A1
公开(公告)日:2022-04-21
申请号:US17384357
申请日:2021-07-23
Applicant: Adobe Inc.
Inventor: Cameron Smith , Ratheesh Kalarot , Wei-An Lin , Richard Zhang , Niloy Mitra , Elya Shechtman , Shabnam Ghadar , Zhixin Shu , Yannick Hold-Geoffrey , Nathan Carr , Jingwan Lu , Oliver Wang , Jun-Yan Zhu
IPC: G06T3/40 , G06F3/0484 , G06N3/08 , G06N3/04
Abstract: An improved system architecture uses a pipeline including a Generative Adversarial Network (GAN) including a generator neural network and a discriminator neural network to generate an image. An input image in a first domain and information about a target domain are obtained. The domains correspond to image styles. An initial latent space representation of the input image is produced by encoding the input image. An initial output image is generated by processing the initial latent space representation with the generator neural network. Using the discriminator neural network, a score is computed indicating whether the initial output image is in the target domain. A loss is computed based on the computed score. The loss is minimized to compute an updated latent space representation. The updated latent space representation is processed with the generator neural network to generate an output image in the target domain.
-
公开(公告)号:US20220121931A1
公开(公告)日:2022-04-21
申请号:US17384371
申请日:2021-07-23
Applicant: Adobe Inc.
Inventor: Ratheesh Kalarot , Wei-An Lin , Cameron Smith , Zhixin Shu , Baldo Faieta , Shabnam Ghadar , Jingwan Lu , Aliakbar Darabi , Jun-Yan Zhu , Niloy Mitra , Richard Zhang , Elya Shechtman
Abstract: Systems and methods train and apply a specialized encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The specialized encoder neural network includes an input layer, a feature extraction layer, and a bottleneck layer positioned after the feature extraction layer. The projection process includes providing an input image to the encoder and producing, by the encoder, a latent space representation of the input image. Producing the latent space representation includes extracting a feature vector from the feature extraction layer, providing the feature vector to the bottleneck layer as input, and producing the latent space representation as output. The latent space representation produced by the encoder is provided as input to the GAN, which generates an output image based upon the latent space representation. The encoder is trained using specialized loss functions including a segmentation loss and a mean latent loss.
-
公开(公告)号:US20210295045A1
公开(公告)日:2021-09-23
申请号:US16822878
申请日:2020-03-18
Applicant: Adobe Inc.
Inventor: Yijun Li , Zhifei Zhang , Richard Zhang , Jingwan Lu
Abstract: The present disclosure relates to systems, computer-implemented methods, and non-transitory computer readable medium for automatically transferring makeup from a reference face image to a target face image using a neural network trained using semi-supervised learning. For example, the disclosed systems can receive, at a neural network, a target face image and a reference face image, where the target face image is selected by a user via a graphical user interface (GUI) and the reference face image has makeup. The systems transfer, by the neural network, the makeup from the reference face image to the target face image, where the neural network is trained to transfer the makeup from the reference face image to the target face image using semi-supervised learning. The systems output for display the makeup on the target face image.
-
公开(公告)号:US12159413B2
公开(公告)日:2024-12-03
申请号:US17693618
申请日:2022-03-14
Applicant: Adobe Inc.
Inventor: Gaurav Parmar , Krishna Kumar Singh , Yijun Li , Richard Zhang , Jingwan Lu
IPC: G06T7/11 , G06T3/4046 , G06T11/00
Abstract: In implementations of systems for image inversion using multiple latent spaces, a computing device implements an inversion system to generate a segment map that segments an input digital image into a first image region and a second image region and assigns the first image region to a first latent space and the second image region to a second latent space that corresponds to a layer of a convolutional neural network. An inverted latent representation of the input digital image is computed using a binary mask for the second image region. The inversion system modifies the inverted latent representation of the input digital image using an edit direction vector that corresponds to a visual feature. An output digital image is generated that depicts a reconstruction of the input digital image having the visual feature based on the modified inverted latent representation of the input digital image.
-
公开(公告)号:US20240338799A1
公开(公告)日:2024-10-10
申请号:US18178212
申请日:2023-03-03
Applicant: Adobe Inc.
Inventor: Yijun Li , Richard Zhang , Krishna Kumar Singh , Jingwan Lu , Gaurav Parmar , Jun-Yan Zhu
IPC: G06T5/00 , G06F40/126 , G06T5/50
CPC classification number: G06T5/70 , G06F40/126 , G06T5/50 , G06T2207/20081 , G06T2207/20084
Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing machine learning models to generate modified digital images. In particular, in some embodiments, the disclosed systems generate image editing directions between textual identifiers of two visual features utilizing a language prediction machine learning model and a text encoder. In some embodiments, the disclosed systems generated an inversion of a digital image utilizing a regularized inversion model to guide forward diffusion of the digital image. In some embodiments, the disclosed systems utilize cross-attention guidance to preserve structural details of a source digital image when generating a modified digital image with a diffusion neural network.
-
28.
公开(公告)号:US20240331236A1
公开(公告)日:2024-10-03
申请号:US18178194
申请日:2023-03-03
Applicant: Adobe Inc.
Inventor: Yijun Li , Richard Zhang , Krishna Kumar Singh , Jingwan Lu , Gaurav Parmar , Jun-Yan Zhu
CPC classification number: G06T11/60 , G06T5/70 , G06T9/00 , G06V10/761 , G06V10/82 , G06V20/70 , G06T2207/20182
Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing machine learning models to generate modified digital images. In particular, in some embodiments, the disclosed systems generate image editing directions between textual identifiers of two visual features utilizing a language prediction machine learning model and a text encoder. In some embodiments, the disclosed systems generated an inversion of a digital image utilizing a regularized inversion model to guide forward diffusion of the digital image. In some embodiments, the disclosed systems utilize cross-attention guidance to preserve structural details of a source digital image when generating a modified digital image with a diffusion neural network.
-
公开(公告)号:US20240281924A1
公开(公告)日:2024-08-22
申请号:US18171046
申请日:2023-02-17
Applicant: ADOBE INC.
Inventor: Taesung Park , Minguk Kang , Richard Zhang , Junyan Zhu , Elya Shechtman , Sylvain Paris
IPC: G06T3/40
CPC classification number: G06T3/4046 , G06T3/4053
Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure obtain a low-resolution image and a text description of the low-resolution image. A mapping network generates a style vector representing the text description of the low-resolution image. An adaptive convolution component generates an adaptive convolution filter based on the style vector. An image generation network generates a high-resolution image corresponding to the low-resolution image based on the adaptive convolution filter.
-
公开(公告)号:US20240169621A1
公开(公告)日:2024-05-23
申请号:US18056579
申请日:2022-11-17
Applicant: ADOBE INC.
Inventor: Yotam Nitzan , Taesung Park , Michaël Gharbi , Richard Zhang , Junyan Zhu , Elya Shechtman
IPC: G06T11/60 , G06V10/774
CPC classification number: G06T11/60 , G06V10/774 , G06T2200/24 , G06V10/82
Abstract: Systems and methods for image generation include obtaining an input image and an attribute value representing an attribute of the input image to be modified; computing a modified latent vector for the input image by applying the attribute value to a basis vector corresponding to the attribute in a latent space of an image generation network; and generating a modified image based on the modified latent vector using the image generation network, wherein the modified image includes the attribute based on the attribute value.
-
-
-
-
-
-
-
-
-