公开(公告)号：US20240169488A1

公开(公告)日：2024-05-23

申请号：US18056405

申请日：2022-11-17

Applicant: ADOBE INC.

Inventor： Nan Liu ,  Yijun Li ,  Michaël Yanis Gharbi ,  Jingwan Lu

IPC: G06T5/00 ,  G06T3/40

CPC classification number: G06T5/002 ,  G06T3/4046 ,  G06T2207/20064 ,  G06T2207/20084

Abstract: Systems and methods for synthesizing images with increased high-frequency detail are described. Embodiments are configured to identify an input image including a noise level and encode the input image to obtain image features. A diffusion model reduces a resolution of the image features at an intermediate stage of the model using a wavelet transform to obtain reduced image features at a reduced resolution, and generates an output image based on the reduced image features using the diffusion model. In some cases, the output image comprises a version of the input image that has a reduced noise level compared to the noise level of the input image.

公开(公告)号：US11907839B2

公开(公告)日：2024-02-20

申请号：US17468511

申请日：2021-09-07

Applicant: Adobe Inc.

Inventor： Ratheesh Kalarot ,  Kevin Wampler ,  Jingwan Lu ,  Jakub Fiser ,  Elya Shechtman ,  Aliakbar Darabi ,  Alexandru Vasile Costin

IPC: G06N3/08 ,  G06F3/04845 ,  G06T11/60 ,  G06T3/40 ,  G06T3/00 ,  G06F3/04847 ,  G06N20/20 ,  G06T5/00 ,  G06T5/20 ,  G06T11/00 ,  G06F18/40 ,  G06F18/211 ,  G06F18/214 ,  G06F18/21 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06F18/211 ,  G06F18/214 ,  G06F18/2163 ,  G06F18/40 ,  G06N3/045 ,  G06N20/20 ,  G06T3/0006 ,  G06T3/0093 ,  G06T3/40 ,  G06T3/4038 ,  G06T3/4046 ,  G06T5/005 ,  G06T5/20 ,  G06T11/001 ,  G06T11/60 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2210/22

Abstract: Systems and methods combine an input image with an edited image generated using a generator neural network to preserve detail from the original image. A computing system provides an input image to a machine learning model to generate a latent space representation of the input image. The system provides the latent space representation to a generator neural network to generate a generated image. The system generates multiple scale representations of the input image, as well as multiple scale representations of the generated image. The system generates a first combined image based on first scale representations of the images and a first value. The system generates a second combined image based on second scale representations of the images and a second value. The system blends the first combined image with the second combined image to generate an output image.

公开(公告)号：US20240037922A1

公开(公告)日：2024-02-01

申请号：US17815451

申请日：2022-07-27

Applicant: Adobe Inc.

Inventor： Yijun Li ,  Nicholas Kolkin ,  Jingwan Lu ,  Elya Shechtman

IPC: G06V10/82 ,  G06V10/77 ,  G06V10/46

CPC classification number: G06V10/82 ,  G06V10/7715 ,  G06V10/469

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for adapting generative neural networks to target domains utilizing an image translation neural network. In particular, in one or more embodiments, the disclosed systems utilize an image translation neural network to translate target results to a source domain for input in target neural network adaptation. For instance, in some embodiments, the disclosed systems compare a translated target result with a source result from a pretrained source generative neural network to adjust parameters of a target generative neural network to produce results corresponding in features to source results and corresponding in style to the target domain.

公开(公告)号：US11875221B2

公开(公告)日：2024-01-16

申请号：US17468476

申请日：2021-09-07

Applicant: Adobe Inc.

Inventor： Wei-An Lin ,  Baldo Faieta ,  Cameron Smith ,  Elya Shechtman ,  Jingwan Lu ,  Jun-Yan Zhu ,  Niloy Mitra ,  Ratheesh Kalarot ,  Richard Zhang ,  Shabnam Ghadar ,  Zhixin Shu

IPC: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06T11/60 ,  G06T3/40 ,  G06N20/20 ,  G06T5/00 ,  G06T5/20 ,  G06T3/00 ,  G06T11/00 ,  G06F18/40 ,  G06F18/211 ,  G06F18/214 ,  G06F18/21 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06F18/211 ,  G06F18/214 ,  G06F18/2163 ,  G06F18/40 ,  G06N3/045 ,  G06N20/20 ,  G06T3/0006 ,  G06T3/0093 ,  G06T3/40 ,  G06T3/4038 ,  G06T3/4046 ,  G06T5/005 ,  G06T5/20 ,  G06T11/001 ,  G06T11/60 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2210/22

Abstract: Systems and methods generate a filtering function for editing an image with reduced attribute correlation. An image editing system groups training data into bins according to a distribution of a target attribute. For each bin, the system samples a subset of the training data based on a pre-determined target distribution of a set of additional attributes in the training data. The system identifies a direction in the sampled training data corresponding to the distribution of the target attribute to generate a filtering vector for modifying the target attribute in an input image, obtains a latent space representation of an input image, applies the filtering vector to the latent space representation of the input image to generate a filtered latent space representation of the input image, and provides the filtered latent space representation as input to a neural network to generate an output image with a modification to the target attribute.

公开(公告)号：US20230360299A1

公开(公告)日：2023-11-09

申请号：US18224916

申请日：2023-07-21

Applicant: Adobe Inc.

Inventor： Yang Yang ,  Zhixin Shu ,  Shabnam Ghadar ,  Jingwan Lu ,  Jakub Fiser ,  Elya Schechtman ,  Cameron Y. Smith ,  Baldo Antonio Faieta ,  Alex Charles Filipkowski

IPC: G06T11/60 ,  G06F21/62 ,  G06F16/56 ,  G06F16/532

CPC classification number: G06T11/60 ,  G06F21/6254 ,  G06F16/56 ,  G06F16/532 ,  G06T2200/24

Abstract: Face anonymization techniques are described that overcome conventional challenges to generate an anonymized face. In one example, a digital object editing system is configured to generate an anonymized face based on a target face and a reference face. As part of this, the digital object editing system employs an encoder as part of machine learning to extract a target encoding of the target face image and a reference encoding of the reference face. The digital object editing system then generates a mixed encoding from the target and reference encodings. The mixed encoding is employed by a machine-learning model of the digital object editing system to generate a mixed face. An object replacement module is used by the digital object editing system to replace the target face in the target digital image with the mixed face.

公开(公告)号：US20230360180A1

公开(公告)日：2023-11-09

申请号：US17661985

申请日：2022-05-04

Applicant: Adobe Inc.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jingwan Lu ,  Scott Cohen ,  Elya Shechtman ,  Connelly Barnes ,  Jianming Zhang ,  Ning Xu ,  Sohrab Amirghodsi

IPC: G06V10/40 ,  G06T3/40 ,  G06T5/00

CPC classification number: G06T5/005 ,  G06T3/4046 ,  G06V10/40 ,  G06T2207/20084

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing a cascaded modulation inpainting neural network. For example, the disclosed systems utilize a cascaded modulation inpainting neural network that includes cascaded modulation decoder layers. For example, in one or more decoder layers, the disclosed systems start with global code modulation that captures the global-range image structures followed by an additional modulation that refines the global predictions. Accordingly, in one or more implementations, the image inpainting system provides a mechanism to correct distorted local details. Furthermore, in one or more implementations, the image inpainting system leverages fast Fourier convolutions block within different resolution layers of the encoder architecture to expand the receptive field of the encoder and to allow the network encoder to better capture global structure.

公开(公告)号：US20230342884A1

公开(公告)日：2023-10-26

申请号：US17725818

申请日：2022-04-21

Applicant: Adobe Inc.

Inventor： Krishna Kumar Singh ,  Yuheng Li ,  Yijun Li ,  Jingwan Lu ,  Elya Shechtman

IPC: G06T5/00 ,  G06V10/82 ,  G06V10/74 ,  G06N3/04

CPC classification number: G06T5/002 ,  G06V10/82 ,  G06V10/761 ,  G06N3/0454 ,  G06T2207/20081

Abstract: An image inpainting system is described that receives an input image that includes a masked region. From the input image, the image inpainting system generates a synthesized image that depicts an object in the masked region by selecting a first code that represents a known factor characterizing a visual appearance of the object and a second code that represents an unknown factor characterizing the visual appearance of the object apart from the known factor in latent space. The input image, the first code, and the second code are provided as input to a generative adversarial network that is trained to generate the synthesized image using contrastive losses. Different synthesized images are generated from the same input image using different combinations of first and second codes, and the synthesized images are output for display.

公开(公告)号：US20230245266A1

公开(公告)日：2023-08-03

申请号：US18298630

申请日：2023-04-11

Applicant: Adobe Inc.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jingwan Lu ,  Scott Cohen ,  Jianming Zhang ,  Ning Su

IPC: G06T3/00 ,  G06T9/00 ,  G06T11/00 ,  G06V10/46 ,  G06V30/24 ,  G06F18/213

CPC classification number: G06T3/0093 ,  G06T9/002 ,  G06T11/00 ,  G06V10/46 ,  G06V30/2504 ,  G06F18/213 ,  G06T2210/36

Abstract: This disclosure describes one or more implementations of a digital image semantic layout manipulation system that generates refined digital images resembling the style of one or more input images while following the structure of an edited semantic layout. For example, in various implementations, the digital image semantic layout manipulation system builds and utilizes a sparse attention warped image neural network to generate high-resolution warped images and a digital image layout neural network to enhance and refine the high-resolution warped digital image into a realistic and accurate refined digital image.

公开(公告)号：US20230162407A1

公开(公告)日：2023-05-25

申请号：US17455796

申请日：2021-11-19

Applicant: ADOBE INC.

Inventor： Ratheesh Kalarot ,  Timothy M. Converse ,  Shabnam Ghadar ,  John Thomas Nack ,  Jingwan Lu ,  Elya Shechtman ,  Baldo Faieta ,  Akhilesh Kumar

IPC: G06T11/00 ,  G06K9/00 ,  G06N3/08

CPC classification number: G06T11/00 ,  G06K9/00288 ,  G06K9/00268 ,  G06N3/08

Abstract: The present disclosure describes systems and methods for image processing. Embodiments of the present disclosure include an image processing apparatus configured to generate modified images (e.g., synthetic faces) by conditionally changing attributes or landmarks of an input image. A machine learning model of the image processing apparatus encodes the input image to obtain a joint conditional vector that represents attributes and landmarks of the input image in a vector space. The joint conditional vector is then modified, according to the techniques described herein, to form a latent vector used to generate a modified image. In some cases, the machine learning model is trained using a generative adversarial network (GAN) with a normalization technique, followed by joint training of a landmark embedding and attribute embedding (e.g., to reduce inference time).

公开(公告)号：US11636570B2

公开(公告)日：2023-04-25

申请号：US17220543

申请日：2021-04-01

Applicant: Adobe Inc.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jingwan Lu ,  Scott Cohen ,  Jianming Zhang ,  Ning Xu

IPC: G06T3/00 ,  G06K9/62 ,  G06T9/00 ,  G06T11/00 ,  G06V10/46 ,  G06V30/24

Abstract: This disclosure describes one or more implementations of a digital image semantic layout manipulation system that generates refined digital images resembling the style of one or more input images while following the structure of an edited semantic layout. For example, in various implementations, the digital image semantic layout manipulation system builds and utilizes a sparse attention warped image neural network to generate high-resolution warped images and a digital image layout neural network to enhance and refine the high-resolution warped digital image into a realistic and accurate refined digital image.