公开(公告)号：US20230122623A1

公开(公告)日：2023-04-20

申请号：US17503671

申请日：2021-10-18

Applicant: Adobe Inc.

Inventor： He Zhang ,  Jeya Maria Jose Valanarasu ,  Jianming Zhang ,  Jose Ignacio Echevarria Vallespi ,  Kalyan Sunkavalli ,  Yilin Wang ,  Yinglan Ma ,  Zhe Lin ,  Zijun Wei

IPC: G06T11/60 ,  G06F3/0484 ,  G06K9/46 ,  G06K9/62 ,  G06N3/08

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating harmonized digital images utilizing an object-to-object harmonization neural network. For example, the disclosed systems implement, and learn parameters for, an object-to-object harmonization neural network to combine a style code from a reference object with features extracted from a target object. Indeed, the disclosed systems extract a style code from a reference object utilizing a style encoder neural network. In addition, the disclosed systems generate a harmonized target object by applying the style code of the reference object to a target object utilizing an object-to-object harmonization neural network.

公开(公告)号：US20200349688A1

公开(公告)日：2020-11-05

申请号：US16930736

申请日：2020-07-16

Applicant: Adobe Inc.

Inventor： Chen Fang ,  Zhe Lin ,  Zhaowen Wang ,  Yulun Zhang ,  Yilin Wang ,  Jimei Yang

IPC: G06T5/50 ,  G06T3/40 ,  G06T11/00 ,  G06T3/00 ,  G06T7/11

Abstract: A style of a digital image is transferred to another digital image of arbitrary resolution. A high-resolution (HR) content image is segmented into several low-resolution (LR) patches. The resolution of a style image is matched to have the same resolution as the LR content image patches. Style transfer is then performed on a patch-by-patch basis using, for example, a pair of feature transforms—whitening and coloring. The patch-by-patch style transfer process is then repeated at several increasing resolutions, or scale levels, of both the content and style images. The results of the style transfer at each scale level are incorporated into successive scale levels up to and including the original HR scale. As a result, style transfer can be performed with images having arbitrary resolutions to produce visually pleasing results with good spatial consistency.

公开(公告)号：US20200258204A1

公开(公告)日：2020-08-13

申请号：US16271058

申请日：2019-02-08

Applicant: Adobe Inc.

Inventor： Chen Fang ,  Zhe Lin ,  Zhaowen Wang ,  Yulun Zhang ,  Yilin Wang ,  Jimei Yang

IPC: G06T5/50 ,  G06T3/40 ,  G06T7/11 ,  G06T3/00 ,  G06T11/00

Abstract: A style of a digital image is transferred to another digital image of arbitrary resolution. A high-resolution (HR) content image is segmented into several low-resolution (LR) patches. The resolution of a style image is matched to have the same resolution as the LR content image patches. Style transfer is then performed on a patch-by-patch basis using, for example, a pair of feature transforms—whitening and coloring. The patch-by-patch style transfer process is then repeated at several increasing resolutions, or scale levels, of both the content and style images. The results of the style transfer at each scale level are incorporated into successive scale levels up to and including the original HR scale. As a result, style transfer can be performed with images having arbitrary resolutions to produce visually pleasing results with good spatial consistency.

公开(公告)号：US12165292B2

公开(公告)日：2024-12-10

申请号：US18317547

申请日：2023-05-15

Applicant: Adobe Inc.

Inventor： He Zhang ,  Seyed Morteza Safdarnejad ,  Yilin Wang ,  Zijun Wei ,  Jianming Zhang ,  Salil Tambe ,  Brian Price

IPC: G06T5/75 ,  G06N3/08 ,  G06T3/4046 ,  G06T7/194

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a plurality of neural networks in a multi-branch pipeline to generate image masks for digital images. Specifically, the disclosed system can classify a digital image as a portrait or a non-portrait image. Based on classifying a portrait image, the disclosed system can utilize separate neural networks to generate a first mask portion for a portion of the digital image including a defined boundary region and a second mask portion for a portion of the digital image including a blended boundary region. The disclosed system can generate the mask portion for the blended boundary region by utilizing a trimap generation neural network to automatically generate a trimap segmentation including the blended boundary region. The disclosed system can then merge the first mask portion and the second mask portion to generate an image mask for the digital image.

公开(公告)号：US20230281763A1

公开(公告)日：2023-09-07

申请号：US18317547

申请日：2023-05-15

Applicant: Adobe Inc.

Inventor： He Zhang ,  Seyed Morteza Safdarnejad ,  Yilin Wang ,  Zijun Wei ,  Jianming Zhang ,  Salil Tambe ,  Brian Price

IPC: G06T5/00 ,  G06T7/194 ,  G06N3/08 ,  G06T3/40

CPC classification number: G06T5/004 ,  G06T7/194 ,  G06N3/08 ,  G06T3/4046 ,  G06T2207/20132 ,  G06T2207/20084 ,  G06T2207/20081

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a plurality of neural networks in a multi-branch pipeline to generate image masks for digital images. Specifically, the disclosed system can classify a digital image as a portrait or a non-portrait image. Based on classifying a portrait image, the disclosed system can utilize separate neural networks to generate a first mask portion for a portion of the digital image including a defined boundary region and a second mask portion for a portion of the digital image including a blended boundary region. The disclosed system can generate the mask portion for the blended boundary region by utilizing a trimap generation neural network to automatically generate a trimap segmentation including the blended boundary region. The disclosed system can then merge the first mask portion and the second mask portion to generate an image mask for the digital image.

公开(公告)号：US11651477B2

公开(公告)日：2023-05-16

申请号：US16988055

申请日：2020-08-07

Applicant: Adobe Inc.

Inventor： He Zhang ,  Seyed Morteza Safdarnejad ,  Yilin Wang ,  Zijun Wei ,  Jianming Zhang ,  Salil Tambe ,  Brian Price

IPC: G06T5/00 ,  G06T7/194 ,  G06N3/08 ,  G06T3/40

CPC classification number: G06T5/004 ,  G06N3/08 ,  G06T3/4046 ,  G06T7/194 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20132

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a plurality of neural networks in a multi-branch pipeline to generate image masks for digital images. Specifically, the disclosed system can classify a digital image as a portrait or a non-portrait image. Based on classifying a portrait image, the disclosed system can utilize separate neural networks to generate a first mask portion for a portion of the digital image including a defined boundary region and a second mask portion for a portion of the digital image including a blended boundary region. The disclosed system can generate the mask portion for the blended boundary region by utilizing a trimap generation neural network to automatically generate a trimap segmentation including the blended boundary region. The disclosed system can then merge the first mask portion and the second mask portion to generate an image mask for the digital image.

公开(公告)号：US11593948B2

公开(公告)日：2023-02-28

申请号：US17177595

申请日：2021-02-17

Applicant: Adobe Inc.

Inventor： Qihang Yu ,  Jianming Zhang ,  He Zhang ,  Yilin Wang ,  Zhe Lin ,  Ning Xu

IPC: G06T7/00 ,  G06T7/194 ,  G06T7/136 ,  G06T7/11 ,  G06T3/40 ,  G06T5/00

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize a progressive refinement network to refine alpha mattes generated utilizing a mask-guided matting neural network. In particular, the disclosed systems can use the matting neural network to process a digital image and a coarse guidance mask to generate alpha mattes at discrete neural network layers. In turn, the disclosed systems can use the progressive refinement network to combine alpha mattes and refine areas of uncertainty. For example, the progressive refinement network can combine a core alpha matte corresponding to more certain core regions of a first alpha matte and a boundary alpha matte corresponding to uncertain boundary regions of a second, higher resolution alpha matte. Based on the combination of the core alpha matte and the boundary alpha matte, the disclosed systems can generate a final alpha matte for use in image matting processes.

公开(公告)号：US11232547B2

公开(公告)日：2022-01-25

申请号：US16930736

申请日：2020-07-16

Applicant: Adobe Inc.

Inventor： Chen Fang ,  Zhe Lin ,  Zhaowen Wang ,  Yulun Zhang ,  Yilin Wang ,  Jimei Yang

IPC: G06T11/00 ,  G06T5/50 ,  G06T3/40 ,  G06T7/11 ,  G06T3/00

Abstract: A style of a digital image is transferred to another digital image of arbitrary resolution. A high-resolution (HR) content image is segmented into several low-resolution (LR) patches. The resolution of a style image is matched to have the same resolution as the LR content image patches. Style transfer is then performed on a patch-by-patch basis using, for example, a pair of feature transforms—whitening and coloring. The patch-by-patch style transfer process is then repeated at several increasing resolutions, or scale levels, of both the content and style images. The results of the style transfer at each scale level are incorporated into successive scale levels up to and including the original HR scale. As a result, style transfer can be performed with images having arbitrary resolutions to produce visually pleasing results with good spatial consistency.

公开(公告)号：US20210241111A1

公开(公告)日：2021-08-05

申请号：US16782793

申请日：2020-02-05

Applicant: Adobe Inc.

Inventor： Shikun Liu ,  Zhe Lin ,  Yilin Wang ,  Jianming Zhang ,  Federico Perazzi

IPC: G06N3/08 ,  G06N3/04

Abstract: The present disclosure relates to shaping the architecture of a neural network. For example, the disclosed systems can provide a neural network shaping mechanism for at least one sampling layer of a neural network. The neural network shaping mechanism can include a learnable scaling factor between a sampling rate of the at least one sampling layer and an additional sampling function. The disclosed systems can learn the scaling factor based on a dataset while jointly learning the network weights of the neural network. Based on the learned scaling factor, the disclosed systems can shape the architecture of the neural network by modifying the sampling rate of the at least one sampling layer.

公开(公告)号：US20210232927A1

公开(公告)日：2021-07-29

申请号：US16751897

申请日：2020-01-24

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Yilin Wang ,  Siyuan Qiao ,  Jianming Zhang

IPC: G06N3/08 ,  G06N20/00

Abstract: In some embodiments, an application receives a request to execute a convolutional neural network model. The application determines the computational complexity requirement for the neural network based on the computing resource available on the device. The application further determines the architecture of the convolutional neural network model by determining the locations of down-sampling layers within the convolutional neural network model based on the computational complexity requirement. The application reconfigures the architecture of the convolutional neural network model by moving the down-sampling layers to the determined locations and executes the convolutional neural network model to generate output results.