公开(公告)号：US20220309762A1

公开(公告)日：2022-09-29

申请号：US17805289

申请日：2022-06-03

Applicant: Adobe Inc.

Inventor： Handong Zhao ,  Zhe Lin ,  Sheng Li ,  Mingyang Ling ,  Jiuxiang Gu

IPC: G06V10/26 ,  G06N3/08 ,  G06K9/62 ,  G06N3/04 ,  G06V10/426

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating semantic scene graphs for digital images using an external knowledgebase for feature refinement. For example, the disclosed system can determine object proposals and subgraph proposals for a digital image to indicate candidate relationships between objects in the digital image. The disclosed system can then extract relationships from an external knowledgebase for refining features of the object proposals and the subgraph proposals. Additionally, the disclosed system can generate a semantic scene graph for the digital image based on the refined features of the object/subgraph proposals. Furthermore, the disclosed system can update/train a semantic scene graph generation network based on the generated semantic scene graph. The disclosed system can also reconstruct the image using object labels based on the refined features to further update/train the semantic scene graph generation network.

公开(公告)号：US20220058777A1

公开(公告)日：2022-02-24

申请号：US16997364

申请日：2020-08-19

Applicant: Adobe Inc.

Inventor： Scott David Cohen ,  Zhihong Ding ,  Zhe Lin ,  Mingyang Ling ,  Luis Angel Figueroa

IPC: G06T5/00 ,  G06K9/46 ,  G06K9/00 ,  G06K9/62

Abstract: Systems, methods, and software are described herein for removing people distractors from images. A distractor mitigation solution implemented in one or more computing devices detects people in an image and identifies salient regions in the image. The solution then determines a saliency cue for each person and classifies each person as wanted or as an unwanted distractor based at least on the saliency cue. An unwanted person is then removed from the image or otherwise reduced from the perspective of being an unwanted distraction.

公开(公告)号：US20210216824A1

公开(公告)日：2021-07-15

申请号：US17215067

申请日：2021-03-29

Applicant: Adobe Inc.

Inventor： Mingyang Ling ,  Alex Filipkowski ,  Zhe Lin ,  Jianming Zhang ,  Samarth Gulati

IPC: G06K9/62 ,  G06N3/04 ,  G06T7/143 ,  G06T7/174 ,  G06T7/136

Abstract: Techniques are disclosed for characterizing and defining the location of a copy space in an image. A methodology implementing the techniques according to an embodiment includes applying a regression convolutional neural network (CNN) to an image. The regression CNN is configured to predict properties of the copy space such as size and type (natural or manufactured). The prediction is conditioned on a determination of the presence of the copy space in the image. The method further includes applying a segmentation CNN to the image. The segmentation CNN is configured to generate one or more pixel-level masks to define the location of copy spaces in the image, whether natural or manufactured, or to define the location of a background region of the image. The segmentation CNN may include a first stage comprising convolutional layers and a second stage comprising pairs of boundary refinement layers and bilinear up-sampling layers.

公开(公告)号：US20200272822A1

公开(公告)日：2020-08-27

申请号：US16874114

申请日：2020-05-14

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xiaohui Shen ,  Mingyang Ling ,  Jianming Zhang ,  Jason Wen Yong Kuen

IPC: G06K9/00 ,  G06K9/62 ,  G06N3/04

Abstract: In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes.

公开(公告)号：US20200210763A1

公开(公告)日：2020-07-02

申请号：US16817234

申请日：2020-03-12

Applicant: ADOBE INC.

Inventor： Zhe Lin ,  Xiaohui Shen ,  Mingyang Ling ,  Jianming Zhang ,  Jason Kuen ,  Brett Butterfield

IPC: G06K9/62 ,  G06K9/52 ,  G06T7/73

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for utilizing a deep neural network-based model to identify similar digital images for query digital images. For example, the disclosed systems utilize a deep neural network-based model to analyze query digital images to generate deep neural network-based representations of the query digital images. In addition, the disclosed systems can generate results of visually-similar digital images for the query digital images based on comparing the deep neural network-based representations with representations of candidate digital images. Furthermore, the disclosed systems can identify visually similar digital images based on user-defined attributes and image masks to emphasize specific attributes or portions of query digital images.

公开(公告)号：US12093306B2

公开(公告)日：2024-09-17

申请号：US18191651

申请日：2023-03-28

Applicant: Adobe Inc.

Inventor： Scott Cohen ,  Zhe Lin ,  Mingyang Ling

IPC: G06V20/70 ,  G06F16/535 ,  G06F18/2113 ,  G06F18/24 ,  G06V10/20 ,  G06V10/764 ,  G06V10/82 ,  G06V20/10

CPC classification number: G06F16/535 ,  G06F18/2113 ,  G06F18/24 ,  G06V10/255 ,  G06V10/764 ,  G06V10/82 ,  G06V20/10 ,  G06V20/70

Abstract: The present disclosure relates to an object selection system that accurately detects and optionally automatically selects user-requested objects (e.g., query objects) in digital images. For example, the object selection system builds and utilizes an object selection pipeline to determine which object detection neural network to utilize to detect a query object based on analyzing the object class of a query object. In particular, the object selection system can identify both known object classes as well as objects corresponding to unknown object classes.

公开(公告)号：US12020414B2

公开(公告)日：2024-06-25

申请号：US17819845

申请日：2022-08-15

Applicant: Adobe Inc.

Inventor： Scott Cohen ,  Zhe Lin ,  Mingyang Ling

IPC: G06T7/00 ,  G06F16/583 ,  G06F40/205 ,  G06T7/70 ,  G06T7/90 ,  G06T11/60

CPC classification number: G06T7/0002 ,  G06F16/5838 ,  G06F16/5854 ,  G06F40/205 ,  G06T7/70 ,  G06T7/90 ,  G06T11/60 ,  G06T2207/20084 ,  G06T2210/12

Abstract: The present disclosure relates to an object selection system that accurately detects and automatically selects target instances of user-requested objects (e.g., a query object instance) in a digital image. In one or more embodiments, the object selection system can analyze one or more user inputs to determine an optimal object attribute detection model from multiple specialized and generalized object attribute models. Additionally, the object selection system can utilize the selected object attribute model to detect and select one or more target instances of a query object in an image, where the image includes multiple instances of the query object.

公开(公告)号：US11797847B2

公开(公告)日：2023-10-24

申请号：US17387195

申请日：2021-07-28

Applicant: Adobe Inc.

Inventor： Scott Cohen ,  Zhe Lin ,  Mingyang Ling

IPC: G06N3/00 ,  G06N3/08 ,  G06T7/11 ,  G06T7/90 ,  G06F16/33 ,  G10L15/22 ,  G06F18/214 ,  G06N3/045 ,  G06V10/25 ,  G06V10/764 ,  G06V10/82

CPC classification number: G06N3/08 ,  G06F16/3344 ,  G06F18/214 ,  G06N3/045 ,  G06T7/11 ,  G06T7/90 ,  G06V10/25 ,  G06V10/764 ,  G06V10/82 ,  G10L15/22 ,  G06T2207/30252 ,  G10L2015/223

Abstract: The systems, methods, a non-transitory computer readable mediums relate to an object selection system that accurately detects and automatically selects user-requested objects (e.g., query objects) in a digital image. For example, the object selection system builds and utilizes an object selection pipeline to determine which object detection neural network to utilize to detect a query object based on analyzing the object class of the query object. In addition, the object selection system can add, update, or replace portions of the object selection pipeline to improve overall accuracy and efficiency of automatic object selection within an image.

公开(公告)号：US20220237826A1

公开(公告)日：2022-07-28

申请号：US17658799

申请日：2022-04-11

Applicant: Adobe Inc.

Inventor： Zhihong Ding ,  Scott Cohen ,  Zhe Lin ,  Mingyang Ling

IPC: G06T7/90 ,  G06K9/62 ,  G06N3/02 ,  G06V10/20

Abstract: The present disclosure relates to a color classification system that accurately classifies objects in digital images based on color. In particular, in one or more embodiments, the color classification system utilizes a multidimensional color space and one or more color mappings to match objects to colors. Indeed, the color classification system can accurately and efficiently detect the color of an object utilizing one or more color similarity regions generated in the multidimensional color space.

公开(公告)号：US11256918B2

公开(公告)日：2022-02-22

申请号：US16874114

申请日：2020-05-14

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xiaohui Shen ,  Mingyang Ling ,  Jianming Zhang ,  Jason Wen Yong Kuen

IPC: G06K9/00 ,  G06K9/62 ,  G06N3/04

Abstract: In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes.