公开(公告)号：US20250078824A1

公开(公告)日：2025-03-06

申请号：US18814275

申请日：2024-08-23

Applicant: Samsung Electronics Co., Ltd.

Inventor： Euisung Kim ,  Yun Tang ,  Taeyeon Ki ,  Divya Neelagiri ,  Vijendra Raj Apsingekar

IPC: G10L15/183 ,  G10L15/06

Abstract: A method includes receiving an utterance from an audio input device. The method also includes determining a context associated with the utterance. The method also includes providing the utterance as an input to a joint model for automatic speech recognition (ASR) and spoken language understanding (SLU), wherein the joint model operates in a single mode to perform both ASR and SLU or a dual mode to perform one of ASR or SLU depending on the context. The method also includes using an output of the joint model to perform an action requested in the utterance. The joint model is trained by training a shared encoder and a shared decoder using a text-to-text task and, after training the shared encoder and the shared decoder, training a speech encoder and the shared encoder using a speech self-supervised learning (SSL) learning task and a text-to-text task with a masked prediction loss.

公开(公告)号：US20240304179A1

公开(公告)日：2024-09-12

申请号：US18596406

申请日：2024-03-05

Applicant: Samsung Electronics Co., Ltd.

Inventor： Euisung Kim ,  Aditya Jajodia ,  Cindy Sushen Tseng ,  Divya Neelagiri ,  Taeyeon Ki ,  Vijendra Raj Apsingekar

IPC: G10L15/06 ,  G10L25/30

CPC classification number: G10L15/063 ,  G10L25/30

Abstract: A method includes receiving, by an automatic speech recognition (ASR)-based spoken language understanding (SLU) model, an input utterance using an audio input device. The method also includes, for each token of the input utterance, generating, using a shared ASR encoder of the ASR-based SLU model, an acoustic representation of acoustic features of the token (the shared ASR encoder including a first adapter layer); determining, using an ASR decoder of the ASR-based SLU model, a text representation of the token using the acoustic representation and any previous tokens (the ASR decoder including a second adapter layer); combining, using a fusion model of the ASR-based SLU model, the text representation and the acoustic representation to generate a joint representation, and determining, using an SLU decoder of the ASR-based SLU model, a semantic label associated with the token based on the joint representation and any previous semantic labels.