公开(公告)号：US20220253990A1

公开(公告)日：2022-08-11

申请号：US17172744

申请日：2021-02-10

Applicant: ADOBE INC.

Inventor： Akhilesh Kumar ,  Zhe Lin ,  Baldo Faieta

IPC: G06T5/20 ,  G06N3/04 ,  G06K9/62 ,  G06T7/00

Abstract: The present disclosure describes systems and methods for image enhancement. Embodiments of the present disclosure provide an image enhancement system with a feedback mechanism that provides quantifiable image enhancement information. An image enhancement system may include a discriminator network that determines the quality of the media object. In cases where the discriminator network determines that the media object has a low image quality score (e.g., an image quality score below a quality threshold), the image enhancement system may perform enhancement on the media object using an enhancement network (e.g., using an enhancement network that includes a generative neural network or a generative adversarial network (GAN) model). The discriminator network may then generate an enhancement score for the enhanced media object that may be provided to the user as a feedback mechanism (e.g., where the enhancement score generated by the discriminator network quantifies the enhancement performed by the enhancement network).

公开(公告)号：US11334971B2

公开(公告)日：2022-05-17

申请号：US16928340

申请日：2020-07-14

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xin Lu ,  Xiaohui Shen ,  Jimei Yang ,  Jiahui Yu

IPC: G06T5/00 ,  G06K9/62

Abstract: Digital image completion by learning generation and patch matching jointly is described. Initially, a digital image having at least one hole is received. This holey digital image is provided as input to an image completer formed with a dual-stage framework that combines a coarse image neural network and an image refinement network. The coarse image neural network generates a coarse prediction of imagery for filling the holes of the holey digital image. The image refinement network receives the coarse prediction as input, refines the coarse prediction, and outputs a filled digital image having refined imagery that fills these holes. The image refinement network generates refined imagery using a patch matching technique, which includes leveraging information corresponding to patches of known pixels for filtering patches generated based on the coarse prediction. Based on this, the image completer outputs the filled digital image with the refined imagery.

公开(公告)号：US20220148285A1

公开(公告)日：2022-05-12

申请号：US17584170

申请日：2022-01-25

Applicant: Adobe Inc.

Inventor： I-Ming Pao ,  Zhe Lin ,  Sarah Stuckey ,  Jianming Zhang ,  Betty Leong

IPC: G06V10/26 ,  G06V10/82 ,  G06T3/40 ,  G06V10/46

Abstract: The present disclosure relates to systems, method, and computer readable media that iteratively apply a neural network to a digital image at a reduced resolution to automatically identify pixels of salient objects portrayed within the digital image. For example, the disclosed systems can generate a reduced-resolution digital image from an input digital image and apply a neural network to identify a region corresponding to a salient object. The disclosed systems can then iteratively apply the neural network to additional reduced-resolution digital images (based on the identified region) to generate one or more reduced-resolution segmentation maps that roughly indicate pixels of the salient object. In addition, the systems described herein can perform post-processing based on the reduced-resolution segmentation map(s) and the input digital image to accurately determine pixels that correspond to the salient object.

公开(公告)号：US20220138185A1

公开(公告)日：2022-05-05

申请号：US17087943

申请日：2020-11-03

Applicant: Adobe Inc.

Inventor： Quan Tran ,  Zhe Lin ,  Xuanli He ,  Walter Chang ,  Trung Bui ,  Franck Dernoncourt

IPC: G06F16/242 ,  G06F40/56 ,  G06N3/08 ,  G06N3/04

Abstract: Systems and methods for natural language processing are described. Embodiments are configured to receive a structured representation of a search query, wherein the structured representation comprises a plurality of nodes and at least one edge connecting two of the nodes, receive a modification expression for the search query, wherein the modification expression comprises a natural language expression, generate a modified structured representation based on the structured representation and the modification expression using a neural network configured to combine structured representation features and natural language expression features, and perform a search based on the modified structured representation.

公开(公告)号：US20210406302A1

公开(公告)日：2021-12-30

申请号：US16910440

申请日：2020-06-24

Applicant: Adobe Inc.

Inventor： Akhilesh Kumar ,  Zhe Lin ,  Ratheesh Kalarot ,  Jinrong Xie ,  Jianming Zhang ,  Baldo Antonio Faieta ,  Alex Charles Filipkowski

IPC: G06F16/55 ,  G06F16/538 ,  G06N20/20 ,  G06N3/02

Abstract: Multidimensional digital content search techniques are described that support an ability of a computing device to perform search with increased granularity and flexibility over conventional techniques. In one example, a control is implemented by a computing device that defines a multidimensional (e.g., two-dimensional) continuous space. Locations in the multidimensional continuous space are usable to different search criteria through different weights applied to the criteria associated with the axes. Therefore, user interaction with this control may be used to define a location and corresponding coordinates that may act as weights to the search criteria in order to perform a search of digital content through use of a single user input.

公开(公告)号：US11195048B2

公开(公告)日：2021-12-07

申请号：US16750478

申请日：2020-01-23

Applicant: Adobe Inc.

Inventor： Trung Huu Bui ,  Zhe Lin ,  Hao Tan ,  Franck Dernoncourt ,  Mohit Bansal

IPC: G06K9/62 ,  G06F40/10 ,  G06K9/46

Abstract: In implementations of generating descriptions of image relationships, a computing device implements a description system which receives a source digital image and a target digital image. The description system generates a source feature sequence from the source digital image and a target feature sequence from the target digital image. A visual relationship between the source digital image and the target digital image is determined by using cross-attention between the source feature sequence and the target feature sequence. The system generates a description of a visual transformation between the source digital image and the target digital image based on the visual relationship.

公开(公告)号：US20210357684A1

公开(公告)日：2021-11-18

申请号：US15930539

申请日：2020-05-13

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Zhe Lin ,  Yilin Wang ,  Tianshu Yu ,  Connelly Barnes ,  Elya Shechtman

IPC: G06K9/62 ,  G06N3/08 ,  G06N20/00 ,  G06F17/18

Abstract: A panoptic labeling system includes a modified panoptic labeling neural network (“modified PLNN”) that is trained to generate labels for pixels in an input image. The panoptic labeling system generates modified training images by combining training images with mask instances from annotated images. The modified PLNN determines a set of labels representing categories of objects depicted in the modified training images. The modified PLNN also determines a subset of the labels representing categories of objects depicted in the input image. For each mask pixel in a modified training image, the modified PLNN calculates a probability indicating whether the mask pixel has the same label as an object pixel. The modified PLNN generates a mask label for each mask pixel, based on the probability. The panoptic labeling system provides the mask label to, for example, a digital graphics editing system that uses the labels to complete an infill operation.

公开(公告)号：US20210350504A1

公开(公告)日：2021-11-11

申请号：US17379622

申请日：2021-07-19

Applicant: ADOBE INC.

Inventor： Xiaohui Shen ,  Zhe Lin ,  Xin Lu ,  Sarah Aye Kong ,  I-Ming Pao ,  Yingcong Chen

IPC: G06T5/00 ,  G06N3/08 ,  G06K9/62 ,  G06K9/66

Abstract: Methods and systems are provided for generating enhanced image. A neural network system is trained where the training includes training a first neural network that generates enhanced images conditioned on content of an image undergoing enhancement and training a second neural network that designates realism of the enhanced images generated by the first neural network. The neural network system is trained by determine loss and accordingly adjusting the appropriate neural network(s). The trained neural network system is used to generate an enhanced aesthetic image from a selected image where the output enhanced aesthetic image has increased aesthetics when compared to the selected image.

公开(公告)号：US20210319056A1

公开(公告)日：2021-10-14

申请号：US16843218

申请日：2020-04-08

Applicant: ADOBE INC.

Inventor： Handong Zhao ,  Ajinkya Kale ,  Xiaowei Jia ,  Zhe Lin

IPC: G06F16/43 ,  G06F16/45 ,  G06F16/438 ,  G06N3/04 ,  G06N3/08

Abstract: The present disclosure relates to a retrieval method including: generating a graph representing a set of users, items, and queries; generating clusters from the media items; generating embeddings for each cluster from embeddings of the items within the corresponding cluster; generating augmented query embeddings for each cluster from the embedding of the corresponding cluster and query embeddings of the queries; inputting the cluster embeddings and the augmented query embeddings to a layer of a graph convolutional network (GCN) to determine user embeddings of the users; inputting the embedding of the given user and a query embedding of the given query to a layer of the GCN to determine a user-specific query embedding; generating a score for each of the items based on the item embeddings and the user-specific query embedding; and presenting the items having the score exceeding a threshold.

公开(公告)号：US20210272253A1

公开(公告)日：2021-09-02

申请号：US16803332

申请日：2020-02-27

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Vipul Dalal ,  Vera Lychagina ,  Shabnam Ghadar ,  Saeid Motiian ,  Rohith mohan Dodle ,  Prethebha Chandrasegaran ,  Mina Doroudi ,  Midhun Harikumar ,  Kannan Iyer ,  Jayant Kumar ,  Gaurav Kukal ,  Daniel Miranda ,  Charles R. McKinney ,  Archit Kalra

IPC: G06T5/50 ,  G06K9/00 ,  G06T7/11 ,  G06K9/62 ,  G06T7/70

Abstract: The present disclosure relates to an image merging system that automatically and seamlessly detects and merges missing people for a set of digital images into a composite group photo. For instance, the image merging system utilizes a number of models and operations to automatically analyze multiple digital images to identify a missing person from a base image, segment the missing person from the second image, and generate a composite group photo by merging the segmented image of the missing person into the base image. In this manner, the image merging system automatically creates merged group photos that appear natural and realistic.