公开(公告)号：US20240037717A1

公开(公告)日：2024-02-01

申请号：US17815409

申请日：2022-07-27

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Lingzhi Zhang ,  Zhe Lin ,  Elya Shechtman ,  Yuqian Zhou ,  Connelly Barnes

IPC: G06T5/00 ,  G06T7/194

CPC classification number: G06T5/005 ,  G06T7/194 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating neural network based perceptual artifact segmentations in synthetic digital image content. The disclosed system utilizing neural networks to detect perceptual artifacts in digital images in connection with generating or modifying digital images. The disclosed system determines a digital image including one or more synthetically modified portions. The disclosed system utilizes an artifact segmentation machine-learning model to detect perceptual artifacts in the synthetically modified portion(s). The artifact segmentation machine-learning model is trained to detect perceptual artifacts based on labeled artifact regions of synthetic training digital images. Additionally, the disclosed system utilizes the artifact segmentation machine-learning model in an iterative inpainting process. The disclosed system utilizes one or more digital image inpainting models to inpaint in a digital image. The disclosed system utilizes the artifact segmentation machine-learning model detect perceptual artifacts in the inpainted portions for additional inpainting iterations.

公开(公告)号：US11868889B2

公开(公告)日：2024-01-09

申请号：US17588516

申请日：2022-01-31

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xiaohui Shen ,  Mingyang Ling ,  Jianming Zhang ,  Jason Wen Yong Kuen

IPC: G06N3/08 ,  G06N3/04 ,  G06V20/20 ,  G06V20/64 ,  G06V10/82 ,  G06V20/10 ,  G06F18/214 ,  G06V10/764 ,  G06V10/44

CPC classification number: G06N3/08 ,  G06F18/214 ,  G06N3/04 ,  G06V10/454 ,  G06V10/764 ,  G06V10/82 ,  G06V20/10 ,  G06V20/20 ,  G06V20/64

Abstract: In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes.

公开(公告)号：US20230418861A1

公开(公告)日：2023-12-28

申请号：US17809503

申请日：2022-06-28

Applicant: Adobe Inc.

Inventor： Zhifei Zhang ,  Zhe Lin

IPC: G06F16/535 ,  G06F16/532 ,  G06F16/33

CPC classification number: G06F16/535 ,  G06F16/532 ,  G06F16/3334

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, in one or more embodiments, the disclosed systems receive an input digital image and search input and further modify the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and the disclosed systems retrieve the image search results utilizing a weighted combination of the queries. In some implementations, the disclosed systems generate an input embedding for the search input (e.g., the multi-modal search input) and retrieve the image search results using the input embedding.

公开(公告)号：US11854119B2

公开(公告)日：2023-12-26

申请号：US17155570

申请日：2021-01-22

Applicant: Adobe Inc.

Inventor： Siavash Khodadadeh ,  Zhe Lin ,  Shabnam Ghadar ,  Saeid Motiian ,  Richard Zhang ,  Ratheesh Kalarot ,  Baldo Faieta

IPC: G06N3/08 ,  G06T7/90 ,  G06N3/045 ,  G06T11/00

CPC classification number: G06T11/001 ,  G06N3/045 ,  G06N3/08 ,  G06T7/90

Abstract: Embodiments are disclosed for automatic object re-colorization in images. In some embodiments, a method of automatic object re-colorization includes receiving a request to recolor an object in an image, the request including an object identifier and a color identifier, identifying an object in the image associated with the object identifier, generating a mask corresponding to the object in the image, providing the image, the mask, and the color identifier to a color transformer network, the color transformer network trained to recolor objects in input images, and generating, by the color transformer network, a recolored image, wherein the object in the recolored image has been recolored to a color corresponding to the color identifier.

公开(公告)号：US20230401827A1

公开(公告)日：2023-12-14

申请号：US17806097

申请日：2022-06-09

Applicant: ADOBE INC.

Inventor： Jason Wen Yong Kuen ,  Dat Ba Huynh ,  Zhe Lin ,  Jiuxiang Gu

IPC: G06V10/774 ,  G06V10/26 ,  G06V10/75 ,  G06V10/77 ,  G06V10/776 ,  G06V10/82

CPC classification number: G06V10/774 ,  G06V10/26 ,  G06V10/759 ,  G06V10/7715 ,  G06V10/776 ,  G06V10/82

Abstract: Systems and methods for image segmentation are described. Embodiments of the present disclosure receive a training image and a caption for the training image, wherein the caption includes text describing an object in the training image; generate a pseudo mask for the object using a teacher network based on the text describing the object; generate a mask for the object using a student network; compute noise information for the training image using a noise estimation network; and update parameters of the student network based on the mask, the pseudo mask, and the noise information.

公开(公告)号：US20230401716A1

公开(公告)日：2023-12-14

申请号：US17806312

申请日：2022-06-10

Applicant: ADOBE INC.

Inventor： Yilin Wang ,  Chenglin Yang ,  Jianming Zhang ,  He Zhang ,  Zijun Wei ,  Zhe Lin

IPC: G06T7/10 ,  G06T11/60 ,  G06N3/08

CPC classification number: G06T7/10 ,  G06T11/60 ,  G06N3/088

Abstract: Systems and methods for image segmentation are described. Embodiments of the present disclosure receive an image depicting an object; generate image features for the image by performing a convolutional self-attention operation that outputs a plurality of attention-weighted values for a convolutional kernel applied at a position of a sliding window on the image; and generate label data that identifies the object based on the image features.

公开(公告)号：US11823490B2

公开(公告)日：2023-11-21

申请号：US17341778

申请日：2021-06-08

Applicant: ADOBE INC.

Inventor： Ratheesh Kalarot ,  Siavash Khodadadeh ,  Baldo Faieta ,  Shabnam Ghadar ,  Saeid Motiian ,  Wei-An Lin ,  Zhe Lin

IPC: G06V40/16 ,  G06N3/084 ,  G06T11/60 ,  G06N3/045

CPC classification number: G06V40/169 ,  G06N3/045 ,  G06N3/084 ,  G06T11/60

Abstract: Systems and methods for image processing are described. One or more embodiments of the present disclosure identify a latent vector representing an image of a face, identify a target attribute vector representing a target attribute for the image, generate a modified latent vector using a mapping network that converts the latent vector and the target attribute vector into a hidden representation having fewer dimensions than the latent vector, wherein the modified latent vector is generated based on the hidden representation, and generate a modified image based on the modified latent vector, wherein the modified image represents the face with the target attribute.

公开(公告)号：US11816181B2

公开(公告)日：2023-11-14

申请号：US17190197

申请日：2021-03-02

Applicant: ADOBE INC.

Inventor： Aashish Misraa ,  Zhe Lin

IPC: G06V10/774 ,  G06F18/214 ,  G06T7/00 ,  G06F18/241 ,  G06V10/82 ,  G06V10/70

CPC classification number: G06F18/214 ,  G06F18/241 ,  G06T7/0002 ,  G06V10/774 ,  G06V10/82 ,  G06V10/87 ,  G06T2207/20081 ,  G06T2207/30168

Abstract: Systems and methods for image processing are described. Embodiments identify a training set including a first image that includes a ground truth blur classification and second image that includes a ground truth blur map, generate a first embedded representation of the first image and a second embedded representation of the second image using an image encoder, predict a blur classification of the first image based on the first embedded representation using a classification layer, predict a blur map of the second image based on the second embedded representation using a map decoder, compute a classification loss based on the predicted blur classification and the ground truth blur classification, train the image encoder and the classification layer based on the classification loss, compute a map loss based on the blur map and the ground truth blur map, and train the image encoder and the map decoder.

公开(公告)号：US11790650B2

公开(公告)日：2023-10-17

申请号：US16998876

申请日：2020-08-20

Applicant: Adobe Inc.

Inventor： Quan Hung Tran ,  Long Thanh Mai ,  Zhe Lin ,  Zhuowan Li

IPC: G06V20/30 ,  G06F16/55 ,  G06F16/535 ,  G06V10/82 ,  G06F40/205 ,  G06V10/75 ,  G06F18/214

CPC classification number: G06V20/30 ,  G06F16/535 ,  G06F16/55 ,  G06F18/214 ,  G06F40/205 ,  G06V10/751 ,  G06V10/82

Abstract: A group captioning system includes computing hardware, software, and/or firmware components in support of the enhanced group captioning contemplated herein. In operation, the system generates a target embedding for a group of target images, as well as a reference embedding for a group of reference images. The system identifies information in-common between the group of target images and the group of reference images and removes the joint information from the target embedding and the reference embedding. The result is a contrastive group embedding that includes a contrastive target embedding and a contrastive reference embedding with which to construct a contrastive group embedding, which is then input to a model to obtain a group caption for the target group of images.

公开(公告)号：US11790234B2

公开(公告)日：2023-10-17

申请号：US18063851

申请日：2022-12-09

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Siyuan Qiao ,  Jianming Zhang

IPC: G06K9/00 ,  G06N3/082 ,  G06N3/04 ,  G06F18/214

CPC classification number: G06N3/082 ,  G06N3/04 ,  G06F18/2148

Abstract: In implementations of resource-aware training for neural network, one or more computing devices of a system implement an architecture optimization module for monitoring parameter utilization while training a neural network. Dead neurons of the neural network are identified as having activation scales less than a threshold. Neurons with activation scales greater than or equal to the threshold are identified as survived neurons. The dead neurons are converted to reborn neurons by adding the dead neurons to layers of the neural network having the survived neurons. The reborn neurons are prevented from connecting to the survived neurons for training the reborn neurons.