公开(公告)号：US20240161462A1

公开(公告)日：2024-05-16

申请号：US18053556

申请日：2022-11-08

Applicant: ADOBE INC.

Inventor： Yosef Gandelsman ,  Taesung Park ,  Richard Zhang ,  Elya Shechtman

IPC: G06V10/774 ,  G06T5/00 ,  G06T11/00 ,  G06V10/776 ,  G06V10/82 ,  G06V10/94

CPC classification number: G06V10/774 ,  G06T5/002 ,  G06T11/00 ,  G06V10/776 ,  G06V10/82 ,  G06V10/945 ,  G06T2200/24 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure include obtaining an image and a prompt for editing the image. A diffusion model is tuned based on the image to generate different versions of the image. The prompt is then encoded to obtain a guidance vector, and the diffusion model generates a modified image based on the image and the encoded text prompt.

公开(公告)号：US20240161327A1

公开(公告)日：2024-05-16

申请号：US18052658

申请日：2022-11-04

Applicant: ADOBE INC.

Inventor： Yinbo Chen ,  Michaël Gharbi ,  Oliver Wang ,  Richard Zhang ,  Elya Shechtman

IPC: G06T7/70 ,  G06T3/40 ,  G06T5/00 ,  G06T7/10

CPC classification number: G06T7/70 ,  G06T3/40 ,  G06T5/003 ,  G06T7/10 ,  G06T2207/20084 ,  G06T2207/20132 ,  G06T2207/20212

Abstract: Aspects of the methods, apparatus, non-transitory computer readable medium, and systems include obtaining a noise map and a global image code encoded from an original image and representing semantic content of the original image; generating a plurality of image patches based on the noise map and the global image code using a diffusion model; and combining the plurality of image patches to produce an output image including the semantic content.

公开(公告)号：US11875221B2

公开(公告)日：2024-01-16

申请号：US17468476

申请日：2021-09-07

Applicant: Adobe Inc.

Inventor： Wei-An Lin ,  Baldo Faieta ,  Cameron Smith ,  Elya Shechtman ,  Jingwan Lu ,  Jun-Yan Zhu ,  Niloy Mitra ,  Ratheesh Kalarot ,  Richard Zhang ,  Shabnam Ghadar ,  Zhixin Shu

IPC: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06T11/60 ,  G06T3/40 ,  G06N20/20 ,  G06T5/00 ,  G06T5/20 ,  G06T3/00 ,  G06T11/00 ,  G06F18/40 ,  G06F18/211 ,  G06F18/214 ,  G06F18/21 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06F18/211 ,  G06F18/214 ,  G06F18/2163 ,  G06F18/40 ,  G06N3/045 ,  G06N20/20 ,  G06T3/0006 ,  G06T3/0093 ,  G06T3/40 ,  G06T3/4038 ,  G06T3/4046 ,  G06T5/005 ,  G06T5/20 ,  G06T11/001 ,  G06T11/60 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2210/22

Abstract: Systems and methods generate a filtering function for editing an image with reduced attribute correlation. An image editing system groups training data into bins according to a distribution of a target attribute. For each bin, the system samples a subset of the training data based on a pre-determined target distribution of a set of additional attributes in the training data. The system identifies a direction in the sampled training data corresponding to the distribution of the target attribute to generate a filtering vector for modifying the target attribute in an input image, obtains a latent space representation of an input image, applies the filtering vector to the latent space representation of the input image to generate a filtered latent space representation of the input image, and provides the filtered latent space representation as input to a neural network to generate an output image with a modification to the target attribute.

公开(公告)号：US20230260175A1

公开(公告)日：2023-08-17

申请号：US17650957

申请日：2022-02-14

Applicant: Adobe Inc.

Inventor： Nadav Epstein ,  Alexei A. Efros ,  Taesung Park ,  Richard Zhang ,  Elya Shechtman

IPC: G06T11/60 ,  G06T7/90

CPC classification number: G06T11/60 ,  G06T7/90 ,  G06T2207/20084 ,  G06T2207/20212

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating digital images depicting photorealistic scenes utilizing a digital image collaging neural network. For example, the disclosed systems utilize a digital image collaging neural network having a particular architecture for disentangling generation of scene layouts and pixel colors for different regions of a digital image. In some cases, the disclosed systems break down the process of generating a collage digital into generating images representing different regions such as a background and a foreground to be collaged into a final result. For example, utilizing the digital image collaging neural network, the disclosed systems determine scene layouts and pixel colors for both foreground digital images and background digital images to ultimately collage the foreground and background together into a collage digital image depicting a real-world scene.

公开(公告)号：US11625875B2

公开(公告)日：2023-04-11

申请号：US17091416

申请日：2020-11-06

Applicant: Adobe Inc.

Inventor： Taesung Park ,  Alexei A. Efros ,  Elya Shechtman ,  Richard Zhang ,  Junyan Zhu

IPC: G06T11/60 ,  G06T7/10 ,  G06N3/088 ,  G06N3/04

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating modified digital images utilizing a novel swapping autoencoder that incorporates scene layout. In particular, the disclosed systems can receive a scene layout map that indicates or defines locations for displaying specific digital content within a digital image. In addition, the disclosed systems can utilize the scene layout map to guide combining portions of digital image latent code to generate a modified digital image with a particular textural appearance and a particular geometric structure defined by the scene layout map. Additionally, the disclosed systems can utilize a scene layout map that defines a portion of a digital image to modify by, for instance, adding new digital content to the digital image, and can generate a modified digital image depicting the new digital content.

公开(公告)号：US20220148241A1

公开(公告)日：2022-05-12

申请号：US17091416

申请日：2020-11-06

Applicant: Adobe Inc.

Inventor： Taesung Park ,  Alexei A. Efros ,  Elya Shechtman ,  Richard Zhang ,  Junyan Zhu

IPC: G06T11/60 ,  G06N3/04 ,  G06T7/10 ,  G06N3/08

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating modified digital images utilizing a novel swapping autoencoder that incorporates scene layout. In particular, the disclosed systems can receive a scene layout map that indicates or defines locations for displaying specific digital content within a digital image. In addition, the disclosed systems can utilize the scene layout map to guide combining portions of digital image latent code to generate a modified digital image with a particular textural appearance and a particular geometric structure defined by the scene layout map. Additionally, the disclosed systems can utilize a scene layout map that defines a portion of a digital image to modify by, for instance, adding new digital content to the digital image, and can generate a modified digital image depicting the new digital content.

公开(公告)号：US20220076374A1

公开(公告)日：2022-03-10

申请号：US17013332

申请日：2020-09-04

Applicant: Adobe Inc.

Inventor： Yijun Li ,  Richard Zhang ,  Jingwan Lu ,  Elya Schechtman

IPC: G06T3/00 ,  G06N20/00

Abstract: One example method involves operations for receiving a request to transform an input image into a target image. Operations further include providing the input image to a machine learning model trained to adapt images. Training the machine learning model includes accessing training data having a source domain of images and a target domain of images with a target style. Training further includes using a pre-trained generative model to generate an adapted source domain of adapted images having the target style. The adapted source domain is generated by determining a rate of change for parameters of the target style, generating weighted parameters by applying a weight to each of the parameters based on their respective rate of change, and applying the weighted parameters to the source domain. Additionally, operations include using the machine learning model to generate the target image by modifying parameters of the input image using the target style.

公开(公告)号：US12254545B2

公开(公告)日：2025-03-18

申请号：US18298138

申请日：2023-04-10

Applicant: Adobe Inc.

Inventor： Taesung Park ,  Alexei A Efros ,  Elya Shechtman ,  Richard Zhang ,  Junyan Zhu

IPC: G06T11/60 ,  G06N3/045 ,  G06N3/088 ,  G06T7/10

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating modified digital images utilizing a novel swapping autoencoder that incorporates scene layout. In particular, the disclosed systems can receive a scene layout map that indicates or defines locations for displaying specific digital content within a digital image. In addition, the disclosed systems can utilize the scene layout map to guide combining portions of digital image latent code to generate a modified digital image with a particular textural appearance and a particular geometric structure defined by the scene layout map. Additionally, the disclosed systems can utilize a scene layout map that defines a portion of a digital image to modify by, for instance, adding new digital content to the digital image, and can generate a modified digital image depicting the new digital content.

公开(公告)号：US12230014B2

公开(公告)日：2025-02-18

申请号：US17680906

申请日：2022-02-25

Applicant: ADOBE INC.

Inventor： Yijun Li ,  Utkarsh Ojha ,  Richard Zhang ,  Jingwan Lu ,  Elya Shechtman ,  Alexei A. Efros

IPC: G06V10/774 ,  G06F3/04842

Abstract: An image generation system enables user input during the process of training a generative model to influence the model's ability to generate new images with desired visual features. A source generative model for a source domain is fine-tuned using training images in a target domain to provide an adapted generative model for the target domain. Interpretable factors are determined for the source generative model and the adapted generative model. A user interface is provided that enables a user to select one or more interpretable factors. The user-selected interpretable factor(s) are used to generate a user-adapted generative model, for instance, by using a loss function based on the user-selected interpretable factor(s). The user-adapted generative model can be used to create new images in the target domain.

公开(公告)号：US12136151B2

公开(公告)日：2024-11-05

申请号：US17650957

申请日：2022-02-14

Applicant: Adobe Inc.

Inventor： Nadav Epstein ,  Alexei A. Efros ,  Taesung Park ,  Richard Zhang ,  Elya Shechtman

IPC: G06T11/60 ,  G06T7/90

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating digital images depicting photorealistic scenes utilizing a digital image collaging neural network. For example, the disclosed systems utilize a digital image collaging neural network having a particular architecture for disentangling generation of scene layouts and pixel colors for different regions of a digital image. In some cases, the disclosed systems break down the process of generating a collage digital into generating images representing different regions such as a background and a foreground to be collaged into a final result. For example, utilizing the digital image collaging neural network, the disclosed systems determine scene layouts and pixel colors for both foreground digital images and background digital images to ultimately collage the foreground and background together into a collage digital image depicting a real-world scene.