公开(公告)号：US20240355022A1

公开(公告)日：2024-10-24

申请号：US18476504

申请日：2023-09-28

Applicant: ADOBE INC.

Inventor： Jing Shi ,  Wei Xiong ,  Zhe Lin ,  Hyun Joon Jung

IPC: G06T11/60 ,  G06T7/194 ,  G06T9/00

CPC classification number: G06T11/60 ,  G06T7/194 ,  G06T9/00 ,  G06T2207/20081

Abstract: One or more aspects of a method, apparatus, and non-transitory computer readable medium include obtaining an input description and an input image depicting a subject, encoding the input description using a text encoder of an image generation model to obtain a text embedding, and encoding the input image using a subject encoder of the image generation model to obtain a subject embedding. A guidance embedding is generated by combining the subject embedding and the text embedding, and then an output image is generated based on the guidance embedding using a diffusion model of the image generation model. The output image depicts aspects of the subject and the input description.

公开(公告)号：US20240161240A1

公开(公告)日：2024-05-16

申请号：US18053027

申请日：2022-11-07

Applicant: Adobe Inc.

Inventor： He Zhang ,  Hyun Joon Jung

IPC: G06T5/50 ,  G06T7/11 ,  G06T7/194 ,  G06V10/26 ,  G06V10/42 ,  G06V10/44 ,  G06V10/82

CPC classification number: G06T5/50 ,  G06T7/11 ,  G06T7/194 ,  G06V10/267 ,  G06V10/42 ,  G06V10/44 ,  G06V10/82 ,  G06T2200/24 ,  G06T2207/20084 ,  G06T2207/20092 ,  G06T2207/20132 ,  G06T2207/20212

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that implement a multi-branch harmonization neural network architecture to harmonize composite images. For example, in one or more implementations, the semantic-guided transformer-based harmonization system uses a convolutional branch, a transformer branch, and a semantic branch to generate a harmonized composite image based on an input composite image and a corresponding segmentation mask. More particularly, the convolutional branch comprises a series of convolutional neural network layers followed by a style normalization layer to extract localized information from the input composite image. Further, the transformer branch comprises a series of transformer neural network layers to extract global information based on different resolutions of the input composite image. The semantic branch includes a visual neural network that generates semantic features that inform the harmonization of the composite images.

公开(公告)号：US12223623B2

公开(公告)日：2025-02-11

申请号：US18053027

申请日：2022-11-07

Applicant: Adobe Inc.

Inventor： He Zhang ,  Hyun Joon Jung

IPC: G06T5/50 ,  G06T7/11 ,  G06T7/194 ,  G06V10/26 ,  G06V10/42 ,  G06V10/44 ,  G06V10/82

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that implement a multi-branch harmonization neural network architecture to harmonize composite images. For example, in one or more implementations, the semantic-guided transformer-based harmonization system uses a convolutional branch, a transformer branch, and a semantic branch to generate a harmonized composite image based on an input composite image and a corresponding segmentation mask. More particularly, the convolutional branch comprises a series of convolutional neural network layers followed by a style normalization layer to extract localized information from the input composite image. Further, the transformer branch comprises a series of transformer neural network layers to extract global information based on different resolutions of the input composite image. The semantic branch includes a visual neural network that generates semantic features that inform the harmonization of the composite images.