公开(公告)号：US20220076693A1

公开(公告)日：2022-03-10

申请号：US17526810

申请日：2021-11-15

Applicant: Adobe Inc.

Inventor： Trung Bui ,  Subhadeep Dey ,  Seunghyun Yoon

IPC: G10L25/63 ,  G06N3/04 ,  G06F17/18 ,  G06F17/16 ,  G10L25/30

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for determining speech emotion. In particular, a speech emotion recognition system generates an audio feature vector and a textual feature vector for a sequence of words. Further, the speech emotion recognition system utilizes a neural attention mechanism that intelligently blends together the audio feature vector and the textual feature vector to generate attention output. Using the attention output, which includes consideration of both audio and text modalities for speech corresponding to the sequence of words, the speech emotion recognition system can apply attention methods to one of the feature vectors to generate a hidden feature vector. Based on the hidden feature vector, the speech emotion recognition system can generate a speech emotion probability distribution of emotions among a group of candidate emotions, and then select one of the candidate emotions as corresponding to the sequence of words.

公开(公告)号：US20210050033A1

公开(公告)日：2021-02-18

申请号：US16543342

申请日：2019-08-16

Applicant: Adobe Inc.

Inventor： Trung Bui ,  Subhadeep Dey ,  Seunghyun Yoon

IPC: G10L25/63 ,  G06N3/04 ,  G10L25/30 ,  G06F17/16 ,  G06F17/18

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for determining speech emotion. In particular, a speech emotion recognition system generates an audio feature vector and a textual feature vector for a sequence of words. Further, the speech emotion recognition system utilizes a neural attention mechanism that intelligently blends together the audio feature vector and the textual feature vector to generate attention output. Using the attention output, which includes consideration of both audio and text modalities for speech corresponding to the sequence of words, the speech emotion recognition system can apply attention methods to one of the feature vectors to generate a hidden feature vector. Based on the hidden feature vector, the speech emotion recognition system can generate a speech emotion probability distribution of emotions among a group of candidate emotions, and then select one of the candidate emotions as corresponding to the sequence of words.

公开(公告)号：US12242820B2

公开(公告)日：2025-03-04

申请号：US17651555

申请日：2022-02-17

Applicant: Adobe Inc.

Inventor： Cesa Salaam ,  Seunghyun Yoon ,  Trung Huu Bui ,  Franck Dernoncourt

IPC: G10L15/22 ,  G06F40/47 ,  G06F40/58 ,  G06N3/045 ,  G06N3/08

Abstract: Techniques for training a language model for code switching content are disclosed. Such techniques include, in some embodiments, generating a dataset, which includes identifying one or more portions within textual content in a first language, the identified one or more portions each including one or more of offensive content or non-offensive content; translating the identified one or more salient portions to a second language; and reintegrating the translated one or more portions into the textual content to generate code-switched textual content. In some cases, the textual content in the first language includes offensive content and non-offensive content, the identified one or more portions include the offensive content, and the translated one or more portions include a translated version of the offensive content. In some embodiments, the code-switched textual content is at least part of a synthetic dataset usable to train a language model, such as a multilingual classification model.

公开(公告)号：US12236975B2

公开(公告)日：2025-02-25

申请号：US17526810

申请日：2021-11-15

Applicant: Adobe Inc.

Inventor： Trung Bui ,  Subhadeep Dey ,  Seunghyun Yoon

IPC: G10L25/00 ,  G06F17/16 ,  G06F17/18 ,  G06N3/047 ,  G10L25/30 ,  G10L25/63

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for determining speech emotion. In particular, a speech emotion recognition system generates an audio feature vector and a textual feature vector for a sequence of words. Further, the speech emotion recognition system utilizes a neural attention mechanism that intelligently blends together the audio feature vector and the textual feature vector to generate attention output. Using the attention output, which includes consideration of both audio and text modalities for speech corresponding to the sequence of words, the speech emotion recognition system can apply attention methods to one of the feature vectors to generate a hidden feature vector. Based on the hidden feature vector, the speech emotion recognition system can generate a speech emotion probability distribution of emotions among a group of candidate emotions, and then select one of the candidate emotions as corresponding to the sequence of words.

公开(公告)号：US12210825B2

公开(公告)日：2025-01-28

申请号：US17455533

申请日：2021-11-18

Applicant: ADOBE INC.

Inventor： Jaemin Cho ,  Seunghyun Yoon ,  Ajinkya Gorakhnath Kale ,  Trung Huu Bui ,  Franck Dernoncourt

IPC: G06F40/253 ,  G06F16/583 ,  G06F18/21 ,  G06F18/214 ,  G06K9/62

Abstract: Systems and methods for image captioning are described. One or more aspects of the systems and methods include generating a training caption for a training image using an image captioning network; encoding the training caption using a multi-modal encoder to obtain an encoded training caption; encoding the training image using the multi-modal encoder to obtain an encoded training image; computing a reward function based on the encoded training caption and the encoded training image; and updating parameters of the image captioning network based on the reward function.

公开(公告)号：US12182524B2

公开(公告)日：2024-12-31

申请号：US17453562

申请日：2021-11-04

Applicant: ADOBE INC.

Inventor： Jianguo Zhang ,  Trung Huu Bui ,  Seunghyun Yoon ,  Xiang Chen ,  Quan Hung Tran ,  Walter W. Chang

IPC: G06F40/40 ,  G06F40/284 ,  G06F40/30 ,  G06V30/19

Abstract: Systems and methods for natural language processing are described. One or more aspects of a method, apparatus, and non-transitory computer readable medium include receiving a text phrase; encoding the text phrase using an encoder to obtain a hidden representation of the text phrase, wherein the encoder is trained during a first training phrase using self-supervised learning based on a first contrastive loss and during a second training phrase using supervised learning based on a second contrastive learning loss; identifying an intent of the text phrase from a predetermined set of intent labels using a classification network, wherein the classification network is jointly trained with the encoder in the second training phase; and generating a response to the text phrase based on the intent.

公开(公告)号：US20230259708A1

公开(公告)日：2023-08-17

申请号：US17650876

申请日：2022-02-14

Applicant: ADOBE INC.

Inventor： Amir Pouran Ben Veyseh ,  Franck Dernoncourt ,  Walter W. Chang ,  Trung Huu Bui ,  Hanieh Deilamsalehy ,  Seunghyun Yoon ,  Rajiv Bhawanji Jain ,  Quan Hung Tran ,  Varun Manjunatha

IPC: G06F40/289 ,  G06F40/30 ,  G10L15/22 ,  G10L15/06 ,  G10L15/16

CPC classification number: G06F40/289 ,  G06F40/30 ,  G10L15/22 ,  G10L15/063 ,  G10L15/16 ,  G10L2015/0635

Abstract: Systems and methods for key-phrase extraction are described. The systems and methods include receiving a transcript including a text paragraph and generating key-phrase data for the text paragraph using a key-phrase extraction network. The key-phrase extraction network is trained to identify domain-relevant key-phrase data based on domain data obtained using a domain discriminator network. The systems and methods further include generating meta-data for the transcript based on the key-phrase data.

公开(公告)号：US20250068924A1

公开(公告)日：2025-02-27

申请号：US18449291

申请日：2023-08-14

Applicant: Adobe Inc.

Inventor： Meryem M'hamdi ,  Seunghyun Yoon ,  Franck Dernoncourt ,  Trung Bui

IPC: G06N3/096 ,  G06F40/30 ,  G06F40/40

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for providing multilingual semantic search results utilizing meta-learning and knowledge distillation. For example, in some implementations, the disclosed systems perform a first inner learning loop for a monolingual to bilingual meta-learning task for a teacher model. Additionally, in some implementations, the disclosed systems perform a second inner learning loop for a bilingual to multilingual meta-learning task for a student model. In some embodiments, the disclosed systems perform knowledge distillation based on the first inner learning loop for the monolingual to bilingual meta-learning task and the second inner learning loop for the bilingual to multilingual meta-learning task. Moreover, in some embodiments, the disclosed systems perform an outer learning loop and update parameters of a deep learning language model based on the first inner learning loop, the second inner learning loop, and the knowledge distillation.

公开(公告)号：US20250028758A1

公开(公告)日：2025-01-23

申请号：US18354833

申请日：2023-07-19

Applicant: Adobe Inc.

Inventor： Seunghyun Yoon

IPC: G06F16/732 ,  G06V10/74 ,  G06V10/776 ,  G06V10/82 ,  G06V10/86 ,  G06V20/40 ,  G06V20/62

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that learns parameters for a natural language video localization model utilizing a curated dataset. In particular, in some embodiments, the disclosed systems generate a set of similarity scores between a target query and a video dataset that includes a plurality of digital videos. For instance, the disclosed systems determines a false-negative threshold by utilizing the set of similarity scores to exclude a subset of false-negative samples from the plurality of digital videos. Further, the disclosed systems determines a negative sample distribution and generates a curated dataset that includes a subset of negative samples with the subset of false-negative samples excluded.

公开(公告)号：US20240355119A1

公开(公告)日：2024-10-24

申请号：US18305587

申请日：2023-04-24

Applicant: ADOBE INC.

Inventor： Ioana Croitoru ,  Trung Huu Bui ,  Zhaowen Wang ,  Seunghyun Yoon ,  Franck Dernoncourt ,  Hailin Jin

IPC: G06V20/40 ,  G06V10/774 ,  G06V20/70 ,  G10L15/04 ,  G10L15/18 ,  G10L15/22 ,  G10L25/57

CPC classification number: G06V20/41 ,  G06V10/774 ,  G06V20/49 ,  G06V20/70 ,  G10L15/04 ,  G10L15/1815 ,  G10L15/22 ,  G10L25/57 ,  G10L15/16

Abstract: One or more aspects of the method, apparatus, and non-transitory computer readable medium include receiving a query relating to a long video. One or more aspects of the method, apparatus, and non-transitory computer readable medium further include generating a segment of the long video corresponding to the query using a machine learning model trained to identify relevant segments from long videos. One or more aspects of the method, apparatus, and non-transitory computer readable medium further include responding to the query based on the generated segment.