公开(公告)号：US11948570B2

公开(公告)日：2024-04-02

申请号：US17654195

申请日：2022-03-09

Applicant: Google LLC

Inventor： Wei Li ,  Rohit Prakash Prabhavalkar ,  Kanury Kanishka Rao ,  Yanzhang He ,  Ian C. Mcgraw ,  Anton Bakhtin

IPC: G10L15/06 ,  G10L15/02 ,  G10L15/16 ,  G10L15/18 ,  G10L15/22 ,  G10L19/00 ,  G10L15/08 ,  G10L15/14

CPC classification number: G10L15/22 ,  G10L15/02 ,  G10L15/063 ,  G10L15/18 ,  G10L19/00 ,  G10L2015/025 ,  G10L2015/088 ,  G10L15/142 ,  G10L2015/223

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detecting utterances of a key phrase in an audio signal. One of the methods includes receiving, by a key phrase spotting system, an audio signal encoding one or more utterances; while continuing to receive the audio signal, generating, by the key phrase spotting system, an attention output using an attention mechanism that is configured to compute the attention output based on a series of encodings generated by an encoder comprising one or more neural network layers; generating, by the key phrase spotting system and using attention output, output that indicates whether the audio signal likely encodes the key phrase; and providing, by the key phrase spotting system, the output that indicates whether the audio signal likely encodes the key phrase.

公开(公告)号：US20230352027A1

公开(公告)日：2023-11-02

申请号：US18219480

申请日：2023-07-07

Applicant: GOOGLE LLC

Inventor： Asaf Aharoni ,  Arun Narayanan ,  Nir Shabat ,  Parisa Haghani ,  Galen Tsai Chuang ,  Yaniv Leviathan ,  Neeraj Gaur ,  Pedro J. Moreno Mengibar ,  Rohit Prakash Prabhavalkar ,  Zhongdi Qu ,  Austin Severn Waters ,  Tomer Amiaz ,  Michiel A.U. Bacchiani

IPC: G10L15/26 ,  H04M3/428 ,  H04M1/663 ,  G10L15/32 ,  H04M3/51 ,  H04M1/02

CPC classification number: G10L15/26 ,  H04M3/4286 ,  H04M1/663 ,  G10L15/32 ,  H04M3/5191 ,  H04M1/02

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for an automated calling system are disclosed. In one aspect, a method includes the actions of receiving audio data of an utterance spoken by a user who is having a telephone conversation with a bot. The actions further include determining a context of the telephone conversation. The actions further include determining a user intent of a first previous portion of the telephone conversation spoken by the user and a bot intent of a second previous portion of the telephone conversation outputted by a speech synthesizer of the bot. The actions further include, based on the audio data of the utterance, the context of the telephone conversation, the user intent, and the bot intent, generating synthesized speech of a reply by the bot to the utterance. The actions further include, providing, for output, the synthesized speech.

公开(公告)号：US20230274736A1

公开(公告)日：2023-08-31

申请号：US18311964

申请日：2023-05-04

Applicant: Google LLC

Inventor： Rohit Prakash Prabhavalkar ,  Golan Pundak ,  Tara N. Sainath ,  Antoine Jean Bruguier

IPC: G10L15/187 ,  G06N20/10 ,  G10L19/04

CPC classification number: G10L15/187 ,  G06N20/10 ,  G10L19/04 ,  G10L2015/088

Abstract: A method of biasing speech recognition includes receiving audio data encoding an utterance and obtaining a set of one or more biasing phrases corresponding to a context of the utterance. Each biasing phrase in the set of one or more biasing phrases includes one or more words. The method also includes processing, using a speech recognition model, acoustic features derived from the audio data and grapheme and phoneme data derived from the set of one or more biasing phrases to generate an output of the speech recognition model. The method also includes determining a transcription for the utterance based on the output of the speech recognition model.

公开(公告)号：US20230237995A1

公开(公告)日：2023-07-27

申请号：US18194586

申请日：2023-03-31

Applicant: Google LLC

Inventor： Rohit Prakash Prabhavalkar ,  Tara N. Sainath ,  Younghui Wu ,  Patrick An Phu Nguyen ,  Zhifeng Chen ,  Chung-Cheng Chiu ,  Anjuli Kannan

IPC: G10L15/197 ,  G10L15/16 ,  G10L15/06 ,  G10L15/02 ,  G10L15/22

CPC classification number: G10L15/197 ,  G10L15/16 ,  G10L15/063 ,  G10L15/02 ,  G10L15/22 ,  G10L2015/025

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-readable storage media, for speech recognition using attention-based sequence-to-sequence models. In some implementations, audio data indicating acoustic characteristics of an utterance is received. A sequence of feature vectors indicative of the acoustic characteristics of the utterance is generated. The sequence of feature vectors is processed using a speech recognition model that has been trained using a loss function that uses a set of speech recognition hypothesis samples, the speech recognition model including an encoder, an attention module, and a decoder. The encoder and decoder each include one or more recurrent neural network layers. A sequence of output vectors representing distributions over a predetermined set of linguistic units is obtained. A transcription for the utterance is obtained based on the sequence of output vectors. Data indicating the transcription of the utterance is provided.

公开(公告)号：US11664021B2

公开(公告)日：2023-05-30

申请号：US17643423

申请日：2021-12-09

Applicant: Google LLC

Inventor： Rohit Prakash Prabhavalkar ,  Golan Pundak ,  Tara N. Sainath ,  Antoine Jean Bruguier

IPC: G10L15/187 ,  G06N20/10 ,  G10L19/04 ,  G10L15/08

CPC classification number: G10L15/187 ,  G06N20/10 ,  G10L19/04 ,  G10L2015/088

Abstract: A method of biasing speech recognition includes receiving audio data encoding an utterance and obtaining a set of one or more biasing phrases corresponding to a context of the utterance. Each biasing phrase in the set of one or more biasing phrases includes one or more words. The method also includes processing, using a speech recognition model, acoustic features derived from the audio data and grapheme and phoneme data derived from the set of one or more biasing phrases to generate an output of the speech recognition model. The method also includes determining a transcription for the utterance based on the output of the speech recognition model.

公开(公告)号：US11158321B2

公开(公告)日：2021-10-26

申请号：US16580726

申请日：2019-09-24

Applicant: Google LLC

Inventor： Asaf Aharoni ,  Arun Narayanan ,  Nir Shabat ,  Parisa Haghani ,  Galen Tsai Chuang ,  Yaniv Leviathan ,  Neeraj Gaur ,  Pedro J. Moreno Mengibar ,  Rohit Prakash Prabhavalkar ,  Zhongdi Qu ,  Austin Severn Waters ,  Tomer Amiaz ,  Michiel A. U. Bacchiani

IPC: G10L15/26 ,  H04M3/428 ,  H04M1/02 ,  G10L15/32 ,  H04M3/51 ,  H04M1/663

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for an automated calling system are disclosed. In one aspect, a method includes the actions of receiving audio data of an utterance spoken by a user who is having a telephone conversation with a bot. The actions further include determining a context of the telephone conversation. The actions further include determining a user intent of a first previous portion of the telephone conversation spoken by the user and a bot intent of a second previous portion of the telephone conversation outputted by a speech synthesizer of the bot. The actions further include, based on the audio data of the utterance, the context of the telephone conversation, the user intent, and the bot intent, generating synthesized speech of a reply by the bot to the utterance. The actions further include, providing, for output, the synthesized speech.

公开(公告)号：US20240420686A1

公开(公告)日：2024-12-19

申请号：US18815200

申请日：2024-08-26

Applicant: Google LLC

Inventor： Rohit Prakash Prabhavalkar ,  Zhifeng Chen ,  Bo Li ,  Chung-Cheng Chiu ,  Kanury Kanishka Rao ,  Yonghui Wu ,  Ron J. Weiss ,  Navdeep Jaitly ,  Michiel A. U. Bacchiani ,  Tara N. Sainath ,  Jan Kazimierz Chorowski ,  Anjuli Patricia Kannan ,  Ekaterina Gonina ,  Patrick An Phu Nguyen

IPC: G10L15/16 ,  G06N3/08 ,  G10L15/02 ,  G10L15/06 ,  G10L15/22 ,  G10L15/26 ,  G10L25/30

Abstract: A method for performing speech recognition using sequence-to-sequence models includes receiving audio data for an utterance and providing features indicative of acoustic characteristics of the utterance as input to an encoder. The method also includes processing an output of the encoder using an attender to generate a context vector, generating speech recognition scores using the context vector and a decoder trained using a training process, and generating a transcription for the utterance using word elements selected based on the speech recognition scores. The transcription is provided as an output of the ASR system.

公开(公告)号：US12027158B2

公开(公告)日：2024-07-02

申请号：US18164923

申请日：2023-02-06

Applicant: Google LLC

Inventor： Ke Hu ,  Tara N. Sainath ,  Ruoming Pang ,  Rohit Prakash Prabhavalkar

IPC: G10L15/18 ,  G06N3/049 ,  G10L15/06 ,  G10L15/16 ,  G10L15/187 ,  G10L19/00

CPC classification number: G10L15/1815 ,  G06N3/049 ,  G10L15/063 ,  G10L15/16 ,  G10L15/187 ,  G10L19/0018

Abstract: A method of performing speech recognition using a two-pass deliberation architecture includes receiving a first-pass hypothesis and an encoded acoustic frame and encoding the first-pass hypothesis at a hypothesis encoder. The first-pass hypothesis is generated by a recurrent neural network (RNN) decoder model for the encoded acoustic frame. The method also includes generating, using a first attention mechanism attending to the encoded acoustic frame, a first context vector, and generating, using a second attention mechanism attending to the encoded first-pass hypothesis, a second context vector. The method also includes decoding the first context vector and the second context vector at a context vector decoder to form a second-pass hypothesis.

公开(公告)号：US20240185841A1

公开(公告)日：2024-06-06

申请号：US18490808

申请日：2023-10-20

Applicant: Google LLC

Inventor： Bo Li ,  Yu Zhang ,  Nanxin Chen ,  Rohit Prakash Prabhavalkar ,  Chao-Han Huck Yang ,  Tara N. Sainath ,  Trevor Strohman

IPC: G10L15/065 ,  G10L15/00

CPC classification number: G10L15/065 ,  G10L15/005

Abstract: A method includes obtaining an ASR model trained to recognize speech in a first language and receiving transcribed training utterances in a second language. The method also includes integrating the ASR model with an input reprogramming module and a latent reprogramming module. The method also includes adapting the ASR model to learn how to recognize speech in the second language by training the input reprogramming module and the latent reprogramming module while parameters of the ASR model are frozen.

公开(公告)号：US20240153498A1

公开(公告)日：2024-05-09

申请号：US18490861

申请日：2023-10-20

Applicant: Google LLC

Inventor： Tara N. Sainath ,  Rohit Prakash Prabhavalkar ,  Diamantino Antonio Caseiro ,  Patrick Maxim Rondon ,  Cyril Allauzen

IPC: G10L15/16 ,  G10L15/06 ,  G10L15/183

CPC classification number: G10L15/16 ,  G10L15/063 ,  G10L15/183

Abstract: A method includes receiving context biasing data that includes a set of unspoken textual utterances corresponding to a particular context. The method also includes obtaining a list of carrier phrases associated with the particular context. For each respective unspoken textual utterance, the method includes generating a corresponding training data pair that includes the respective unspoken textual utterance and a carrier phrase. For each respective training data pair, the method includes tokenizing the respective training data pair into a sequence of sub-word units, generating a first higher order textual feature representation for a corresponding sub-word unit, receiving the first higher order textual feature representation, and generating a first probability distribution over possible text units. The method also includes training a speech recognition model based on the first probability distribution over possible text units.