公开(公告)号：US20240135512A1

公开(公告)日：2024-04-25

申请号：US18190556

申请日：2023-03-27

Applicant: Adobe Inc.

Inventor： Krishna Kumar Singh ,  Yijun Li ,  Jingwan Lu ,  Duygu Ceylan Aksit ,  Yangtuanfeng Wang ,  Jimei Yang ,  Tobias Hinz ,  Qing Liu ,  Jianming Zhang ,  Zhe Lin

IPC: G06T5/00 ,  G06T7/11 ,  G06V10/82 ,  G06V40/10

CPC classification number: G06T5/005 ,  G06T7/11 ,  G06V10/82 ,  G06V40/10 ,  G06T2207/20021 ,  G06T2207/20084 ,  G06T2207/20212 ,  G06T2207/30196

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

公开(公告)号：US20240135511A1

公开(公告)日：2024-04-25

申请号：US18190544

申请日：2023-03-27

Applicant: Adobe Inc.

Inventor： Krishna Kumar Singh ,  Yijun Li ,  Jingwan Lu ,  Duygu Ceylan Aksit ,  Yangtuanfeng Wang ,  Jimei Yang ,  Tobias Hinz ,  Qing Liu ,  Jianming Zhang ,  Zhe Lin

IPC: G06T5/00 ,  G06V10/25 ,  G06V10/44 ,  G06V10/82

CPC classification number: G06T5/005 ,  G06V10/25 ,  G06V10/44 ,  G06V10/82 ,  G06T2207/30196

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

公开(公告)号：US20240127452A1

公开(公告)日：2024-04-18

申请号：US17937680

申请日：2022-10-03

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Haitian Zheng ,  Elya Shechtman ,  Jianming Zhang ,  Jingwan Lu ,  Ning Xu ,  Qing Liu ,  Scott Cohen ,  Sohrab Amirghodsi

IPC: G06T7/11

CPC classification number: G06T7/11 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20132

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.

公开(公告)号：US11948281B2

公开(公告)日：2024-04-02

申请号：US16864388

申请日：2020-05-01

Applicant: ADOBE INC.

Inventor： Zhe Lin ,  Yu Zeng ,  Jimei Yang ,  Jianming Zhang ,  Elya Shechtman

IPC: G06T5/00 ,  G06T3/40 ,  G06T3/4046 ,  G06T3/4076

CPC classification number: G06T5/005 ,  G06T3/4046 ,  G06T3/4076

Abstract: Methods and systems are provided for accurately filling holes, regions, and/or portions of high-resolution images using guided upsampling during image inpainting. For instance, an image inpainting system can apply guided upsampling to an inpainted image result to enable generation of a high-resolution inpainting result from a lower-resolution image that has undergone inpainting. To allow for guided upsampling during image inpainting, one or more neural networks can be used. For instance, a low-resolution result neural network (e.g., comprised of an encoder and a decoder) and a high-resolution input neural network (e.g., comprised of an encoder and a decoder). The image inpainting system can use such networks to generate a high-resolution inpainting image result that fills the hole, region, and/or portion of the image.

公开(公告)号：US11934448B2

公开(公告)日：2024-03-19

申请号：US18302201

申请日：2023-04-18

Applicant: Adobe Inc.

Inventor： Pramod Srinivasan ,  Zhe Lin ,  Samarth Gulati ,  Saeid Motiian ,  Midhun Harikumar ,  Baldo Antonio Faieta ,  Alex C. Filipkowski

IPC: G06F16/532 ,  G06F16/51 ,  G06F16/538 ,  G06F16/54 ,  G06F16/583 ,  G06F40/30

CPC classification number: G06F16/532 ,  G06F16/51 ,  G06F16/538 ,  G06F16/54 ,  G06F16/583 ,  G06F40/30

Abstract: Keyword localization digital image search techniques are described. These techniques support an ability to indicate “where” a corresponding keyword is to be expressed with respect to a layout in a respective digital image resulting from a search query. The search query may also include an indication of a size of the keyword as expressed in the digital image, a number of instances of the keyword, and so forth. Additionally, the techniques and systems as described herein support real time search through use of keyword signatures.

公开(公告)号：US20240004924A1

公开(公告)日：2024-01-04

申请号：US17809781

申请日：2022-06-29

Applicant: Adobe Inc.

Inventor： Zhifei Zhang ,  Zhe Lin ,  Zhihong Ding ,  Scott Cohen ,  Darshan Prasad

IPC: G06F16/538 ,  G06F16/532 ,  G06T7/11 ,  G06T5/50 ,  G06F16/583

CPC classification number: G06F16/538 ,  G06F16/532 ,  G06F16/5838 ,  G06T5/50 ,  G06T7/11

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, in one or more embodiments, the disclosed systems receive an input digital image and search input and further modify the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and the disclosed systems retrieve the image search results utilizing a weighted combination of the queries. In some implementations, the disclosed systems generate an input embedding for the search input (e.g., the multi-modal search input) and retrieve the image search results using the input embedding.

公开(公告)号：US20230419571A1

公开(公告)日：2023-12-28

申请号：US17809494

申请日：2022-06-28

Applicant: Adobe Inc.

Inventor： Zhifei Zhang ,  Zhe Lin ,  Scott Cohen ,  Kevin Gary Smith

IPC: G06T11/60 ,  G06T11/20 ,  G06F16/532

CPC classification number: G06T11/60 ,  G06T11/203 ,  G06F16/532 ,  G06T2200/24 ,  G06T2207/20084 ,  G06F3/0482

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, in one or more embodiments, the disclosed systems receive an input digital image and search input and further modify the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and the disclosed systems retrieve the image search results utilizing a weighted combination of the queries. In some implementations, the disclosed systems generate an input embedding for the search input (e.g., the multi-modal search input) and retrieve the image search results using the input embedding.

公开(公告)号：US20230376828A1

公开(公告)日：2023-11-23

申请号：US17664079

申请日：2022-05-19

Applicant: ADOBE INC.

Inventor： Handong Zhao ,  Haoyu Ma ,  Zhe Lin ,  Ajinkya Gorakhnath Kale ,  Tong Yu ,  Jiuxiang Gu ,  Sunav Choudhary ,  Venkata Naveen Kumar Yadav Marri

IPC: G06N20/00 ,  G06F16/9538 ,  G06Q30/06

CPC classification number: G06N20/00 ,  G06F16/9538 ,  G06Q30/0641

Abstract: Systems and methods for product retrieval are described. One or more aspects of the systems and methods include receiving a query that includes a text description of a product associated with a brand; identifying the product based on the query by comparing the text description to a product embedding of the product, wherein the product embedding is based on a brand embedding of the brand; and displaying product information for the product in response to the query, wherein the product information includes the brand.

公开(公告)号：US20230326028A1

公开(公告)日：2023-10-12

申请号：US17658873

申请日：2022-04-12

Applicant: Adobe Inc.

Inventor： Jianming Zhang ,  Soo Ye Kim ,  Simon Niklaus ,  Yifei Fan ,  Su Chen ,  Zhe Lin

IPC: G06T7/11 ,  G06T7/215 ,  G06T7/50

CPC classification number: G06T7/11 ,  G06T2207/20084 ,  G06T7/50 ,  G06T7/215

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing machine learning models to generate refined depth maps of digital images utilizing digital segmentation masks. In particular, in one or more embodiments, the disclosed systems generate a depth map for a digital image utilizing a depth estimation machine learning model, determine a digital segmentation mask for the digital image, and generate a refined depth map from the depth map and the digital segmentation mask utilizing a depth refinement machine learning model. In some embodiments, the disclosed systems generate first and second intermediate depth maps using the digital segmentation mask and an inverse digital segmentation mask and merger the first and second intermediate depth maps to generate the refined depth map.

公开(公告)号：US20230325992A1

公开(公告)日：2023-10-12

申请号：US17658774

申请日：2022-04-11

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Sijie Zhu ,  Jason Wen Yong Kuen ,  Scott Cohen ,  Zhifei Zhang

IPC: G06T5/50 ,  G06T3/60

CPC classification number: G06T5/50 ,  G06T3/60 ,  G06T7/194

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that utilizes artificial intelligence to learn to recommend foreground object images for use in generating composite images based on geometry and/or lighting features. For instance, in one or more embodiments, the disclosed systems transform a foreground object image corresponding to a background image using at least one of a geometry transformation or a lighting transformation. The disclosed systems further generating predicted embeddings for the background image, the foreground object image, and the transformed foreground object image within a geometry-lighting-sensitive embedding space utilizing a geometry-lighting-aware neural network. Using a loss determined from the predicted embeddings, the disclosed systems update parameters of the geometry-lighting-aware neural network. The disclosed systems further provide a variety of efficient user interfaces for generating composite digital images.