公开(公告)号：US20240304179A1

公开(公告)日：2024-09-12

申请号：US18596406

申请日：2024-03-05

Applicant: Samsung Electronics Co., Ltd.

Inventor： Euisung Kim ,  Aditya Jajodia ,  Cindy Sushen Tseng ,  Divya Neelagiri ,  Taeyeon Ki ,  Vijendra Raj Apsingekar

IPC: G10L15/06 ,  G10L25/30

CPC classification number: G10L15/063 ,  G10L25/30

Abstract: A method includes receiving, by an automatic speech recognition (ASR)-based spoken language understanding (SLU) model, an input utterance using an audio input device. The method also includes, for each token of the input utterance, generating, using a shared ASR encoder of the ASR-based SLU model, an acoustic representation of acoustic features of the token (the shared ASR encoder including a first adapter layer); determining, using an ASR decoder of the ASR-based SLU model, a text representation of the token using the acoustic representation and any previous tokens (the ASR decoder including a second adapter layer); combining, using a fusion model of the ASR-based SLU model, the text representation and the acoustic representation to generate a joint representation, and determining, using an SLU decoder of the ASR-based SLU model, a semantic label associated with the token based on the joint representation and any previous semantic labels.

公开(公告)号：US20240054999A1

公开(公告)日：2024-02-15

申请号：US18297509

申请日：2023-04-07

Applicant: Samsung Electronics Co., Ltd.

Inventor： Cindy Sushen Tseng ,  Srinivasa Rao Ponakala ,  Myungjong Kim ,  Taeyeon Ki ,  Vijendra Raj Apsingekar

IPC: G10L15/22 ,  G10L15/08 ,  G10L25/51

CPC classification number: G10L15/22 ,  G10L15/08 ,  G10L25/51 ,  G10L2015/223 ,  G10L2015/088

Abstract: A method includes obtaining an audio input and a location associated with an electronic device. The method also includes generating an audio embedding associated with the audio input. The method further includes determining a first difference between the audio embedding associated with the audio input and an audio embedding associated with a known user. The method also includes determining a second difference between the location associated with the electronic device and a known location associated with the known user. The method further includes generating, using a false trigger mitigation (FTM) system, a probability of the audio input including a false trigger for automatic speech recognition based on the audio input, the first difference, and the second difference. In addition, the method includes determining whether to perform automatic speech recognition based on the probability.

公开(公告)号：US20240029723A1

公开(公告)日：2024-01-25

申请号：US17937198

申请日：2022-09-30

Applicant: Samsung Electronics Co., Ltd.

Inventor： Sivakumar Balasubramanian ,  Gowtham Srinivasan ,  Srinivasa Rao Ponakala ,  Vijendra Raj Apsingekar ,  Anil Sunder Yadav

IPC: G10L15/197 ,  G10L15/06 ,  G10L15/22

CPC classification number: G10L15/197 ,  G10L15/063 ,  G10L15/22 ,  G10L2015/223

Abstract: A method comprises obtaining an audio input. The method also includes providing at least a portion of the audio input to a frame-level detector model. The method also includes obtaining a first output of the frame-level detector model including frame-level predictions associated with at least the portion of the audio input. The method also includes providing at least one chunked audio frame to a word-level verifier model. The method also includes obtaining a second output of the word-level verifier model including word-level probabilities associated with the at least one chunked audio frame. The method also includes instructing performance of automatic speech recognition on the audio input based on the word-level probabilities associated with the at least one chunked audio frame.

公开(公告)号：US20230419979A1

公开(公告)日：2023-12-28

申请号：US18046041

申请日：2022-10-12

Applicant: Samsung Electronics Co., Ltd.

Inventor： Myungjong Kim ,  Taeyeon Ki ,  Vijendra Raj Apsingekar ,  Sungjae Park ,  SeungBeom Ryu ,  Hyuk Oh

IPC: G10L21/028 ,  G10L17/06 ,  G10L17/02

CPC classification number: G10L21/028 ,  G10L17/06 ,  G10L17/02

Abstract: A method includes obtaining at least a portion of an audio stream containing speech activity. At least the portion of the audio stream includes multiple segments. The method also includes, for each of the multiple segments, generating an embedding vector that represents the segment. The method further includes, within each of multiple local windows, clustering the embedding vectors into one or more clusters to perform speaker identification. Different clusters correspond to different speakers. The method also includes presenting at least one first sequence of speaker identities based on the speaker identification performed for the local windows. The method further includes, within each of multiple global windows, clustering the embedding vectors into one or more clusters to perform speaker identification. Each global window includes two or more local windows. In addition, the method includes presenting at least one second sequence of speaker identities based on the speaker identification performed for the global windows.

公开(公告)号：US12170079B2

公开(公告)日：2024-12-17

申请号：US17444367

申请日：2021-08-03

Applicant: Samsung Electronics Co., Ltd.

Inventor： Divya Neelagiri ,  Taeyeon Ki ,  Vijendra Raj Apsingekar

IPC: G10L15/06 ,  G10L15/18 ,  G10L15/26

Abstract: A method includes training a set of teacher models. Training the set of teacher models includes, for each individual teacher model of the set of teacher models, training the individual teacher model to transcribe unlabeled audio samples and predict a pseudo labeled dataset having multiple labels. At least some of the unlabeled audio samples contain named entity (NE) audio data. At least some of the labels include transcribed NE labels corresponding to the NE audio data. The method also includes correcting at least some of the transcribed NE labels using user-specific NE textual data. The method further includes retraining the set of teacher models based on the pseudo labeled dataset from a selected one of the teacher models, where the selected one of the teacher models predicts the pseudo labeled dataset more accurately than other teacher models of the set of teacher models.

公开(公告)号：US20230368786A1

公开(公告)日：2023-11-16

申请号：US17929280

申请日：2022-09-01

Applicant: Samsung Electronics Co., Ltd.

Inventor： Sivakumar Balasubramanian ,  Gowtham Srinivasan ,  Srinivasa Rao Ponakala ,  Vijendra Raj Apsingekar ,  Anil Sunder Yadav

IPC: G10L15/22 ,  G10L15/06

CPC classification number: G10L15/22 ,  G10L15/063 ,  G10L2015/223 ,  G10L2015/0631

Abstract: A method includes accessing, using at least one processor of an electronic device, a machine learning model. The machine learning model is a trained student model that is trained using audio samples in a plurality of accent types. The method also includes receiving, using the at least one processor, an audio input from an audio input device. The method further includes providing, using the at least one processor, the audio input to the trained student model. The method also includes receiving, using the at least one processor, an output from the trained student model including frame-level probabilities associated with the audio input. In addition, the method includes instructing, using the at least one processor, at least one action based on the frame-level probabilities associated with the audio input.

公开(公告)号：US20230040181A1

公开(公告)日：2023-02-09

申请号：US17444367

申请日：2021-08-03

Applicant: Samsung Electronics Co., Ltd.

Inventor： Divya Neelagiri ,  Taeyeon Ki ,  Vijendra Raj Apsingekar

IPC: G10L15/06 ,  G10L15/26 ,  G10L15/18

Abstract: A method includes training a set of teacher models. Training the set of teacher models includes, for each individual teacher model of the set of teacher models, training the individual teacher model to transcribe unlabeled audio samples and predict a pseudo labeled dataset having multiple labels. At least some of the unlabeled audio samples contain named entity (NE) audio data. At least some of the labels include transcribed NE labels corresponding to the NE audio data. The method also includes correcting at least some of the transcribed NE labels using user-specific NE textual data. The method further includes retraining the set of teacher models based on the pseudo labeled dataset from a selected one of the teacher models, where the selected one of the teacher models predicts the pseudo labeled dataset more accurately than other teacher models of the set of teacher models.

公开(公告)号：US20250078824A1

公开(公告)日：2025-03-06

申请号：US18814275

申请日：2024-08-23

Applicant: Samsung Electronics Co., Ltd.

Inventor： Euisung Kim ,  Yun Tang ,  Taeyeon Ki ,  Divya Neelagiri ,  Vijendra Raj Apsingekar

IPC: G10L15/183 ,  G10L15/06

Abstract: A method includes receiving an utterance from an audio input device. The method also includes determining a context associated with the utterance. The method also includes providing the utterance as an input to a joint model for automatic speech recognition (ASR) and spoken language understanding (SLU), wherein the joint model operates in a single mode to perform both ASR and SLU or a dual mode to perform one of ASR or SLU depending on the context. The method also includes using an output of the joint model to perform an action requested in the utterance. The joint model is trained by training a shared encoder and a shared decoder using a text-to-text task and, after training the shared encoder and the shared decoder, training a speech encoder and the shared encoder using a speech self-supervised learning (SSL) learning task and a text-to-text task with a masked prediction loss.

公开(公告)号：US20240056761A1

公开(公告)日：2024-02-15

申请号：US18335730

申请日：2023-06-15

Applicant: Samsung Electronics Co., Ltd.

Inventor： Vijendra Raj Apsingekar ,  Akash Sahoo ,  Anil S. Yadav ,  Sivakumar Balasubramanian

IPC: H04S7/00 ,  H04S3/00 ,  G06F3/16 ,  G10L19/008

CPC classification number: H04S7/304 ,  H04S3/008 ,  G06F3/165 ,  G10L19/008 ,  H04S2400/11

Abstract: A method includes obtaining video content and associated substantially mono audio content. The method also includes determining at least one of a position or a motion trajectory of each of one or more objects detected in the video content and classifying each of the one or more objects into one of multiple object classes. The method further includes separating audio streams within the audio content based on the video content. Each of the audio streams is associated with one of multiple audio sources. The method also includes classifying each of the audio sources into one of the object classes. In addition, the method includes, for each audio source classified into the same object class as one of the one or more objects, distributing the audio stream associated with that audio source into multiple audio channels based on at least one of the position or the motion trajectory of that object.

公开(公告)号：US20230419958A1

公开(公告)日：2023-12-28

申请号：US17937692

申请日：2022-10-03

Applicant: Samsung Electronics Co., Ltd.

Inventor： Divya Neelagiri ,  Cindy Sushen Tseng ,  Vijendra Raj Apsingekar

IPC: G10L15/197 ,  G10L15/00 ,  G10L15/22

CPC classification number: G10L15/197 ,  G10L15/005 ,  G10L15/22

Abstract: A method includes obtaining an audio input of a person speaking, where the audio input is captured by an electronic device. The method also includes, for each of multiple language types, (i) determining a first probability that the person is speaking in the language type by applying a trained spoken language identification model to the audio input, (ii) determining at least one second probability that the person is speaking in the language type based on at least one characteristic of the person or the electronic device, and (iii) determining a score for the language type based on a weighted sum of the first and second probabilities. The method further includes identifying the language type associated with a highest score as a spoken language of the person in the audio input.