公开(公告)号：US11875510B2

公开(公告)日：2024-01-16

申请号：US17200525

申请日：2021-03-12

Applicant: Adobe Inc.

Inventor： Yilin Wang ,  Chenglin Yang ,  Jianming Zhang ,  He Zhang ,  Zhe Lin

IPC: G06K9/00 ,  G06T7/11 ,  G06N3/08 ,  G06T3/40 ,  G06F18/213 ,  G06N3/044

CPC classification number: G06T7/11 ,  G06F18/213 ,  G06N3/044 ,  G06N3/08 ,  G06T3/4046 ,  G06T2207/20084

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilizes a neural network having a hierarchy of hierarchical point-wise refining blocks to generate refined segmentation masks for high-resolution digital visual media items. For example, in one or more embodiments, the disclosed systems utilize a segmentation refinement neural network having an encoder and a recursive decoder to generate the refined segmentation masks. The recursive decoder includes a deconvolution branch for generating feature maps and a refinement branch for generating and refining segmentation masks. In particular, in some cases, the refinement branch includes a hierarchy of hierarchical point-wise refining blocks that recursively refine a segmentation mask generated for a digital visual media item. In some cases, the disclosed systems utilize a segmentation refinement neural network that includes a low-resolution network and a high-resolution network, each including an encoder and a recursive decoder, to generate the refined segmentation masks.

公开(公告)号：US20240005574A1

公开(公告)日：2024-01-04

申请号：US17810392

申请日：2022-07-01

Applicant: Adobe Inc.

Inventor： Zhifei Zhang ,  Zhe Lin ,  Scott Cohen ,  Darshan Prasad ,  Zhihong Ding

IPC: G06T11/40 ,  G06T5/00 ,  G06T7/12

CPC classification number: G06T11/40 ,  G06T5/005 ,  G06T7/12 ,  G06T2207/20084

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for transferring global style features between digital images utilizing one or more machine learning models or neural networks. In particular, in one or more embodiments, the disclosed systems receive a request to transfer a global style from a source digital image to a target digital image, identify at least one target object within the target digital image, and transfer the global style from the source digital image to the target digital image while maintaining an object style of the at least one target object.

公开(公告)号：US11854244B2

公开(公告)日：2023-12-26

申请号：US18048311

申请日：2022-10-20

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Zhe Lin ,  Yilin Wang ,  Tianshu Yu ,  Connelly Barnes ,  Elya Shechtman

IPC: G06V10/75 ,  G06F17/18 ,  G06N3/08 ,  G06N20/00 ,  G06V10/82 ,  G06F18/214 ,  G06F18/22 ,  G06F18/211 ,  G06F18/213 ,  G06V10/74 ,  G06V10/771 ,  G06V10/774 ,  G06V20/70

CPC classification number: G06V10/757 ,  G06F17/18 ,  G06F18/211 ,  G06F18/213 ,  G06F18/214 ,  G06F18/22 ,  G06N3/08 ,  G06N20/00 ,  G06V10/761 ,  G06V10/771 ,  G06V10/774 ,  G06V10/82 ,  G06V20/70

Abstract: A panoptic labeling system includes a modified panoptic labeling neural network (“modified PLNN”) that is trained to generate labels for pixels in an input image. The panoptic labeling system generates modified training images by combining training images with mask instances from annotated images. The modified PLNN determines a set of labels representing categories of objects depicted in the modified training images. The modified PLNN also determines a subset of the labels representing categories of objects depicted in the input image. For each mask pixel in a modified training image, the modified PLNN calculates a probability indicating whether the mask pixel has the same label as an object pixel. The modified PLNN generates a mask label for each mask pixel, based on the probability. The panoptic labeling system provides the mask label to, for example, a digital graphics editing system that uses the labels to complete an infill operation.

公开(公告)号：US11854206B2

公开(公告)日：2023-12-26

申请号：US17735156

申请日：2022-05-03

Applicant: Adobe Inc.

Inventor： Federico Perazzi ,  Zhe Lin ,  Ping Hu ,  Oliver Wang ,  Fabian David Caba Heilbron

IPC: G06T7/11 ,  G06F17/15 ,  G06V20/40 ,  G06N3/045 ,  G06V10/80

CPC classification number: G06T7/11 ,  G06F17/15 ,  G06N3/045 ,  G06V10/806 ,  G06V20/46 ,  G06V20/49 ,  G06T2207/10016 ,  G06T2207/20084

Abstract: A Video Semantic Segmentation System (VSSS) is disclosed that performs accurate and fast semantic segmentation of videos using a set of temporally distributed neural networks. The VSSS receives as input a video signal comprising a contiguous sequence of temporally-related video frames. The VSSS extracts features from the video frames in the contiguous sequence and based upon the extracted features, selects, from a set of labels, a label to be associated with each pixel of each video frame in the video signal. In certain embodiments, a set of multiple neural networks are used to extract the features to be used for video segmentation and the extraction of features is distributed among the multiple neural networks in the set. A strong feature representation representing the entirety of the features is produced for each video frame in the sequence of video frames by aggregating the output features extracted by the multiple neural networks.

公开(公告)号：US20230401717A1

公开(公告)日：2023-12-14

申请号：US17806314

申请日：2022-06-10

Applicant: ADOBE INC.

Inventor： Yilin Wang ,  Chenglin Yang ,  Jianming Zhang ,  He Zhang ,  Zijun Wei ,  Zhe Lin

IPC: G06T7/11 ,  G06V20/70 ,  G06V10/778 ,  G06V10/82

CPC classification number: G06T7/11 ,  G06V20/70 ,  G06V10/778 ,  G06V10/82 ,  G06T2207/20021 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for image segmentation are described. Embodiments of the present disclosure receive an image depicting an object; generate image features for the image by performing an atrous self-attention operation based on a plurality of dilation rates for a convolutional kernel applied at a position of a sliding window on the image; and generate label data that identifies the object based on the image features.

公开(公告)号：US20230368339A1

公开(公告)日：2023-11-16

申请号：US17663317

申请日：2022-05-13

Applicant: Adobe Inc.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jingwan Lu ,  Scott Cohen ,  Elya Shechtman ,  Connelly Barnes ,  Jianming Zhang ,  Ning Xu ,  Sohrab Amirghodsi

IPC: G06T5/00 ,  G06T7/11 ,  G06N3/04

CPC classification number: G06T5/005 ,  G06T7/11 ,  G06N3/04 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing class-specific cascaded modulation inpainting neural network. For example, the disclosed systems utilize a class-specific cascaded modulation inpainting neural network that includes cascaded modulation decoder layers to generate replacement pixels portraying a particular target object class. To illustrate, in response to user selection of a replacement region and target object class, the disclosed systems utilize a class-specific cascaded modulation inpainting neural network corresponding to the target object class to generate an inpainted digital image that portrays an instance of the target object class within the replacement region. Moreover, in one or more embodiments the disclosed systems train class-specific cascaded modulation inpainting neural networks corresponding to a variety of target object classes, such as a sky object class, a water object class, a ground object class, or a human object class.

公开(公告)号：US11816888B2

公开(公告)日：2023-11-14

申请号：US16853111

申请日：2020-04-20

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xiaohui Shen ,  Jonathan Brandt ,  Jianming Zhang ,  Chen Fang

IPC: G06F16/51 ,  G06F16/28 ,  G06F16/2457 ,  G06F16/583 ,  G06F18/2113 ,  G06F18/21 ,  G06F18/23213 ,  G06F18/2413 ,  G06N3/045 ,  G06N20/10 ,  G06V20/00 ,  G06V10/762 ,  G06V10/764 ,  G06N3/08

CPC classification number: G06V20/35 ,  G06F16/24578 ,  G06F16/285 ,  G06F16/51 ,  G06F16/583 ,  G06F18/217 ,  G06F18/2113 ,  G06F18/23213 ,  G06F18/24147 ,  G06N3/045 ,  G06N3/08 ,  G06V10/763 ,  G06V10/764 ,  G06N20/10

Abstract: Embodiments of the present invention provide an automated image tagging system that can predict a set of tags, along with relevance scores, that can be used for keyword-based image retrieval, image tag proposal, and image tag auto-completion based on user input. Initially, during training, a clustering technique is utilized to reduce cluster imbalance in the data that is input into a convolutional neural network (CNN) for training feature data. In embodiments, the clustering technique can also be utilized to compute data point similarity that can be utilized for tag propagation (to tag untagged images). During testing, a diversity based voting framework is utilized to overcome user tagging biases. In some embodiments, bigram re-weighting can down-weight a keyword that is likely to be part of a bigram based on a predicted tag set.

公开(公告)号：US11809822B2

公开(公告)日：2023-11-07

申请号：US16803480

申请日：2020-02-27

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xihui Liu ,  Quan Tran ,  Jianming Zhang ,  Handong Zhao

IPC: G06F16/538 ,  G06F40/216 ,  G06F16/583 ,  G06N3/08 ,  G06F40/30 ,  G06F16/56 ,  G06F16/2457 ,  G06V30/262 ,  G06F18/22 ,  G06F18/213 ,  G06F18/214 ,  G06V30/19 ,  G06V10/75

CPC classification number: G06F40/216 ,  G06F16/24578 ,  G06F16/538 ,  G06F16/56 ,  G06F16/5854 ,  G06F18/213 ,  G06F18/214 ,  G06F18/22 ,  G06F40/30 ,  G06N3/08 ,  G06V10/75 ,  G06V30/19147 ,  G06V30/274

Abstract: Certain embodiments involve a method for generating a search result. The method includes processing devices performing operations including receiving a query having a text input by a joint embedding model trained to generate an image result. Training the joint embedding model includes accessing a set of images and textual information. Training further includes encoding the images into image feature vectors based on spatial features. Further, training includes encoding the textual information into textual feature vectors based on semantic information. Training further includes generating a set of image-text pairs based on matches between image feature vectors and textual feature vectors. Further, training includes generating a visual grounding dataset based on spatial information. Training further includes generating a set of visual-semantic joint embeddings by grounding the image-text pairs with the visual grounding dataset. Additionally, operations include generating an image result for display by the joint embedding model based on the text input.

公开(公告)号：US11776237B2

公开(公告)日：2023-10-03

申请号：US16997364

申请日：2020-08-19

Applicant: Adobe Inc.

Inventor： Scott David Cohen ,  Zhihong Ding ,  Zhe Lin ,  Mingyang Ling ,  Luis Angel Figueroa

IPC: G06V10/46 ,  G06T5/00 ,  G06V40/10 ,  G06V40/16 ,  G06F18/241

CPC classification number: G06V10/464 ,  G06F18/241 ,  G06T5/005 ,  G06V40/10 ,  G06V40/161 ,  G06T2207/20081 ,  G06T2207/30201

Abstract: Systems, methods, and software are described herein for removing people distractors from images. A distractor mitigation solution implemented in one or more computing devices detects people in an image and identifies salient regions in the image. The solution then determines a saliency cue for each person and classifies each person as wanted or as an unwanted distractor based at least on the saliency cue. An unwanted person is then removed from the image or otherwise reduced from the perspective of being an unwanted distraction.

公开(公告)号：US20230260164A1

公开(公告)日：2023-08-17

申请号：US17651075

申请日：2022-02-15

Applicant: ADOBE INC.

Inventor： Xin Yuan ,  Zhe Lin ,  Jason Wen Yong Kuen ,  Jianming Zhang ,  John Philip Collomosse

IPC: G06T11/00 ,  G06F16/53 ,  G06N20/00

CPC classification number: G06T11/00 ,  G06F16/53 ,  G06N20/00 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for image generation are described. Embodiments of the present disclosure receive a text phrase that describes a target image to be generated; generate text features based on the text phrase; retrieve a search image based on the text phrase; and generate the target image using an image generation network based on the text features and the search image.