公开(公告)号：US10997752B1

公开(公告)日：2021-05-04

申请号：US16813050

申请日：2020-03-09

Applicant: Adobe Inc.

Inventor： Seungjoo Yoo ,  Richard Zhang ,  Matthew Fisher ,  Jingwan Lu

IPC: G06T7/13 ,  G06T11/00 ,  G06T7/90

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing an edge prediction neural network and edge-guided colorization neural network to transform grayscale digital images into colorized digital images. In one or more embodiments, the disclosed systems apply a color edge prediction neural network to a grayscale image to generate a color edge map indicating predicted chrominance edges. The disclosed systems can present the color edge map to a user via a colorization graphical user interface and receive user color points and color edge modifications. The disclosed systems can apply a second neural network, an edge-guided colorization neural network, to the color edge map or a modified edge map, user color points, and the grayscale image to generate an edge-constrained colorized digital image.

公开(公告)号：US20250111565A1

公开(公告)日：2025-04-03

申请号：US18479234

申请日：2023-10-02

Applicant: ADOBE INC.

Inventor： David Charles Epstein ,  Richard Zhang ,  Ishan Kapil Jain

IPC: G06T11/60 ,  G06T7/00

Abstract: A method, apparatus, and non-transitory computer readable medium for obtaining an input image comprising a plurality of pixels. A machine learning model generates annotation information indicating whether each of the plurality of pixels is synthetically generated. A combined image is generated based on the annotation information. In some cases, the combined image shows a synthetically generated region of the input image.

公开(公告)号：US20250045994A1

公开(公告)日：2025-02-06

申请号：US18924508

申请日：2024-10-23

Applicant: Adobe Inc.

Inventor： Nadav Epstein ,  Alexei A Efros ,  Taesung Park ,  Richard Zhang ,  Elya Shechtman

IPC: G06T11/60 ,  G06T7/90

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating digital images depicting photorealistic scenes utilizing a digital image collaging neural network. For example, the disclosed systems utilize a digital image collaging neural network having a particular architecture for disentangling generation of scene layouts and pixel colors for different regions of a digital image. In some cases, the disclosed systems break down the process of generating a collage digital into generating images representing different regions such as a background and a foreground to be collaged into a final result. For example, utilizing the digital image collaging neural network, the disclosed systems determine scene layouts and pixel colors for both foreground digital images and background digital images to ultimately collage the foreground and background together into a collage digital image depicting a real-world scene.

公开(公告)号：US20240185588A1

公开(公告)日：2024-06-06

申请号：US18062314

申请日：2022-12-06

Applicant: ADOBE INC.

Inventor： Nupur Kumari ,  Richard Zhang ,  Junyan Zhu ,  Elya Shechtman

IPC: G06V10/778 ,  G06V10/75 ,  G06V10/774

CPC classification number: G06V10/778 ,  G06V10/751 ,  G06V10/774

Abstract: Systems and methods for fine-tuning diffusion models are described. Embodiments of the present disclosure obtain an input text indicating an element to be included in an image; generate a synthetic image depicting the element based on the input text using a diffusion model trained by comparing synthetic images depicting the element to training images depicting elements similar to the element and updating selected parameters corresponding to an attention layer of the diffusion model based on the comparison.

公开(公告)号：US11854119B2

公开(公告)日：2023-12-26

申请号：US17155570

申请日：2021-01-22

Applicant: Adobe Inc.

Inventor： Siavash Khodadadeh ,  Zhe Lin ,  Shabnam Ghadar ,  Saeid Motiian ,  Richard Zhang ,  Ratheesh Kalarot ,  Baldo Faieta

IPC: G06N3/08 ,  G06T7/90 ,  G06N3/045 ,  G06T11/00

CPC classification number: G06T11/001 ,  G06N3/045 ,  G06N3/08 ,  G06T7/90

Abstract: Embodiments are disclosed for automatic object re-colorization in images. In some embodiments, a method of automatic object re-colorization includes receiving a request to recolor an object in an image, the request including an object identifier and a color identifier, identifying an object in the image associated with the object identifier, generating a mask corresponding to the object in the image, providing the image, the mask, and the color identifier to a color transformer network, the color transformer network trained to recolor objects in input images, and generating, by the color transformer network, a recolored image, wherein the object in the recolored image has been recolored to a color corresponding to the color identifier.

公开(公告)号：US20230316606A1

公开(公告)日：2023-10-05

申请号：US17655739

申请日：2022-03-21

Applicant: Adobe Inc.

Inventor： Hui Qu ,  Baldo Faieta ,  Cameron Smith ,  Elya Shechtman ,  Jingwan Lu ,  Ratheesh Kalarot ,  Richard Zhang ,  Saeid Motiian ,  Shabnam Ghadar ,  Wei-An Lin

IPC: G06N3/04 ,  G06T11/60

CPC classification number: G06T11/60 ,  G06N3/0454

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for latent-based editing of digital images using a generative neural network. In particular, in one or more embodiments, the disclosed systems perform latent-based editing of a digital image by mapping a feature tensor and a set of style vectors for the digital image into a joint feature style space. In one or more implementations, the disclosed systems apply a joint feature style perturbation and/or modification vectors within the joint feature style space to determine modified style vectors and a modified feature tensor. Moreover, in one or more embodiments the disclosed systems generate a modified digital image utilizing a generative neural network from the modified style vectors and the modified feature tensor.

公开(公告)号：US20230316591A1

公开(公告)日：2023-10-05

申请号：US17709895

申请日：2022-03-31

Applicant: Adobe Inc.

Inventor： Zhixin Shu ,  Zhe Lin ,  Yuchen Liu ,  Yijun Li ,  Richard Zhang

IPC: G06T11/00 ,  G06V10/40 ,  G06V10/774

CPC classification number: G06T11/00 ,  G06V10/40 ,  G06V10/7747

Abstract: Techniques for identity preserved controllable facial image manipulation are described that support generation of a manipulated digital image based on a facial image and a render image. For instance, a facial image depicting a facial representation of an individual is received as input. A feature space including an identity parameter and at least one other visual parameter is extracted from the facial image. An editing module edits one or more of the visual parameters and preserves the identity parameter. A renderer generates a render image depicting a morphable model reconstruction of the facial image based on the edit. The render image and facial image are encoded, and a generator of a neural network is implemented to generate a manipulated digital image based on the encoded facial image and the encoded render image.

公开(公告)号：US20230274535A1

公开(公告)日：2023-08-31

申请号：US17680906

申请日：2022-02-25

Applicant: ADOBE INC.

Inventor： Yijun Li ,  Utkarsh Ojha ,  Richard Zhang ,  Jingwan Lu ,  Elya Shechtman ,  Alexei A. Efros

IPC: G06V10/774 ,  G06F3/04842

CPC classification number: G06V10/7747 ,  G06F3/04842

Abstract: An image generation system enables user input during the process of training a generative model to influence the model's ability to generate new images with desired visual features. A source generative model for a source domain is fine-tuned using training images in a target domain to provide an adapted generative model for the target domain. Interpretable factors are determined for the source generative model and the adapted generative model. A user interface is provided that enables a user to select one or more interpretable factors. The user-selected interpretable factor(s) are used to generate a user-adapted generative model, for instance, by using a loss function based on the user-selected interpretable factor(s). The user-adapted generative model can be used to create new images in the target domain.

公开(公告)号：US20230055204A1

公开(公告)日：2023-02-23

申请号：US17405207

申请日：2021-08-18

Applicant: Adobe Inc.

Inventor： Adrian-Stefan Ungureanu ,  Ionut Mironica ,  Richard Zhang

IPC: G06T11/00 ,  G06N3/04

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize one or more stages of a two-stage image colorization neural network to colorize or re-colorize digital images. In one or more embodiments, the disclosed system generates a color digital image from a grayscale digital image by utilizing a colorization neural network. Additionally, the disclosed system receives one or more inputs indicating local hints comprising one or more color selections to apply to one or more objects of the color digital image. The disclosed system then utilizes a re-colorization neural network to generate a modified digital image from the color digital image by modifying one or more colors of the object(s) based on the luminance channel, color channels, and selected color(s).

公开(公告)号：US20220414431A1

公开(公告)日：2022-12-29

申请号：US17899936

申请日：2022-08-31

Applicant: Adobe Inc.

Inventor： Richard Zhang ,  Sylvain Philippe Paris ,  Junyan Zhu ,  Aaron Phillip Hertzmann ,  Jacob Minyoung Huh

IPC: G06N3/04 ,  G06T11/60 ,  G06F17/18

Abstract: A target image is projected into a latent space of generative model by determining a latent vector by applying a gradient-free technique and a class vector by applying a gradient-based technique. An image is generated from the latent and class vectors, and a loss function is used to determine a loss between the target image and the generated image. This determining of the latent vector and the class vector, generating an image, and using the loss function is repeated until a loss condition is satisfied. In response to the loss condition being satisfied, the latent and class vectors that resulted in the loss condition being satisfied are identified as the final latent and class vectors, respectively. The final latent and class vectors are provided to the generative model and multiple weights of the generative model are adjusted to fine-tune the generative model.