公开(公告)号：US12211509B2

公开(公告)日：2025-01-28

申请号：US17821160

申请日：2022-08-19

Applicant: Google LLC

Inventor： Chao Zhang ,  Bo Li ,  Zhiyun Lu ,  Tara N. Sainath ,  Shuo-yiin Chang

IPC: G10L15/30 ,  G06N7/01

Abstract: A speech recognition model includes an encoder network, a prediction network, and a joint network. The encoder network is configured to receive a sequence of acoustic frames characterizing an input utterance; and generate, at each of a plurality of output steps, a higher order feature representation for a corresponding acoustic frame in the sequence of acoustic frames. The prediction network is configured to: receive a sequence of non-blank symbols output by a final Softmax layer; and generate, at each of the plurality of output steps, a dense representation. The joint network is configured to generate, at each of the plurality of output steps based on the higher order feature representation and the dense representation, a probability distribution over possible speech recognition hypotheses. The joint network includes a stack of gating and bilinear pooling to fuse the dense representation and the higher order feature representation.

公开(公告)号：US20240029716A1

公开(公告)日：2024-01-25

申请号：US18480827

申请日：2023-10-04

Applicant: Google LLC

Inventor： Thibault Doutre ,  Wei Han ,  Min Ma ,  Zhiyun Lu ,  Chung-Cheng Chiu ,  Ruoming Pang ,  Arun Narayanan ,  Ananya Misra ,  Yu Zhang ,  Liangliang Cao

IPC: G10L15/06 ,  G10L15/08 ,  G10L15/18 ,  G06N3/045

CPC classification number: G10L15/063 ,  G10L15/083 ,  G10L15/18 ,  G06N3/045

Abstract: A method for training a streaming automatic speech recognition student model includes receiving a plurality of unlabeled student training utterances. The method also includes, for each unlabeled student training utterance, generating a transcription corresponding to the respective unlabeled student training utterance using a plurality of non-streaming automated speech recognition (ASR) teacher models. The method further includes distilling a streaming ASR student model from the plurality of non-streaming ASR teacher models by training the streaming ASR student model using the plurality of unlabeled student training utterances paired with the corresponding transcriptions generated by the plurality of non-streaming ASR teacher models.

公开(公告)号：US11804212B2

公开(公告)日：2023-10-31

申请号：US17348118

申请日：2021-06-15

Applicant: Google LLC

Inventor： Thibault Doutre ,  Wei Han ,  Min Ma ,  Zhiyun Lu ,  Chung-Cheng Chiu ,  Ruoming Pang ,  Arun Narayanan ,  Ananya Misra ,  Yu Zhang ,  Liangliang Cao

IPC: G10L15/06 ,  G10L15/08 ,  G10L15/18 ,  G06N3/04 ,  G06N3/045

CPC classification number: G10L15/063 ,  G06N3/045 ,  G10L15/083 ,  G10L15/18

Abstract: A method for training a streaming automatic speech recognition student model includes receiving a plurality of unlabeled student training utterances. The method also includes, for each unlabeled student training utterance, generating a transcription corresponding to the respective unlabeled student training utterance using a plurality of non-streaming automated speech recognition (ASR) teacher models. The method further includes distilling a streaming ASR student model from the plurality of non-streaming ASR teacher models by training the streaming ASR student model using the plurality of unlabeled student training utterances paired with the corresponding transcriptions generated by the plurality of non-streaming ASR teacher models.

公开(公告)号：US20230343332A1

公开(公告)日：2023-10-26

申请号：US18304064

申请日：2023-04-20

Applicant: Google LLC

Inventor： Ronny Huang ,  Shuo-yiin Chang ,  David Rybach ,  Rohit Prakash Prabhavalkar ,  Tara N. Sainath ,  Cyril Allauzen ,  Charles Caleb Peyser ,  Zhiyun Lu

IPC: G10L15/04 ,  G10L25/93 ,  G10L15/197 ,  G10L15/06 ,  G10L15/22 ,  G10L15/02

CPC classification number: G10L15/197 ,  G10L15/02 ,  G10L15/04 ,  G10L15/063 ,  G10L15/22 ,  G10L25/93 ,  G10L2015/025 ,  G10L2025/932

Abstract: A joint segmenting and ASR model includes an encoder and decoder. The encoder configured to: receive a sequence of acoustic frames characterizing one or more utterances; and generate, at each output step, a higher order feature representation for a corresponding acoustic frame. The decoder configured to: receive the higher order feature representation and generate, at each output step: a probability distribution over possible speech recognition hypotheses, and an indication of whether the corresponding output step corresponds to an end of speech segment. The j oint segmenting and ASR model trained on a set of training samples, each training sample including: audio data characterizing a spoken utterance; and a corresponding transcription of the spoken utterance, the corresponding transcription having an end of speech segment ground truth token inserted into the corresponding transcription automatically based on a set of heuristic-based rules and exceptions applied to the training sample.

公开(公告)号：US20240013777A1

公开(公告)日：2024-01-11

申请号：US18320458

申请日：2023-05-19

Applicant: Google LLC

Inventor： Zhiyun Lu ,  Yu Zhang ,  Wei Han ,  Yongqiang Wang ,  Parisa Haghani ,  Zhehuai Chen

IPC: G10L15/16 ,  G10L15/06

CPC classification number: G10L15/16 ,  G10L15/063

Abstract: A method includes obtaining a corpus of unlabeled training data including a plurality of spoken utterances, each corresponding spoken utterance of the plurality of spoken utterances includes audio data characterizing the corresponding spoken utterance. The method also includes receiving a target domain. The method also includes selecting, using a contrastive data selection model, a subset of the utterances from the corpus of unlabeled training data that correspond to the target domain. The method includes training an automatic speech recognition (ASR) model on the subset of utterances.

公开(公告)号：US20220343894A1

公开(公告)日：2022-10-27

申请号：US17348118

申请日：2021-06-15

Applicant: Google LLC

Inventor： Thibault Doutre ,  Wei Han ,  Min Ma ,  Zhiyun Lu ,  Chung-Cheng Chiu ,  Ruoming Pang ,  Arun Narayanan ,  Ananya Misra ,  Yu Zhang ,  Liangliang Cao

IPC: G10L15/06 ,  G06N3/04 ,  G10L15/18 ,  G10L15/08

Abstract: A method for training a streaming automatic speech recognition student model includes receiving a plurality of unlabeled student training utterances. The method also includes, for each unlabeled student training utterance, generating a transcription corresponding to the respective unlabeled student training utterance using a plurality of non-streaming automated speech recognition (ASR) teacher models. The method further includes distilling a streaming ASR student model from the plurality of non-streaming ASR teacher models by training the streaming ASR student model using the plurality of unlabeled student training utterances paired with the corresponding transcriptions generated by the plurality of non-streaming ASR teacher models.