公开(公告)号：US11881210B2

公开(公告)日：2024-01-23

申请号：US16867427

申请日：2020-05-05

Applicant: Google LLC

Inventor： Tom Marius Kenter ,  Manish Kumar Sharma ,  Robert Andrew James Clark ,  Aliaksei Severyn

IPC: G10L15/16 ,  G06N3/084 ,  G10L15/02 ,  G10L15/06

CPC classification number: G10L15/16 ,  G06N3/084 ,  G10L15/02 ,  G10L15/063 ,  G10L2015/025 ,  G10L2015/027

Abstract: A method for generating a prosodic representation includes receiving a text utterance having one or more words. Each word has at least one syllable having at least one phoneme. The method also includes generating, using a Bidirectional Encoder Representations from Transformers (BERT) model, a sequence of wordpiece embeddings and selecting an utterance embedding for the text utterance, the utterance embedding representing an intended prosody. Each wordpiece embedding is associated with one of the one or more words of the text utterance. For each syllable, using the selected utterance embedding and a prosody model that incorporates the BERT model, the method also includes generating a corresponding prosodic syllable embedding for the syllable based on the wordpiece embedding associated with the word that includes the syllable and predicting a duration of the syllable by encoding linguistic features of each phoneme of the syllable with the corresponding prosodic syllable embedding for the syllable.

公开(公告)号：US20210350795A1

公开(公告)日：2021-11-11

申请号：US16867427

申请日：2020-05-05

Applicant: Google LLC

Inventor： Tom Marius Kenter ,  Manish Kumar Sharma ,  Robert Andrew James Clark ,  Aliaksei Severyn

IPC: G10L15/16 ,  G10L15/02 ,  G06N3/08 ,  G10L15/06

Abstract: A method for generating a prosodic representation includes receiving a text utterance having one or more words. Each word has at least one syllable having at least one phoneme. The method also includes generating, using a Bidirectional Encoder Representations from Transformers (BERT) model, a sequence of wordpiece embeddings and selecting an utterance embedding for the text utterance, the utterance embedding representing an intended prosody. Each wordpiece embedding is associated with one of the one or more words of the text utterance. For each syllable, using the selected utterance embedding and a prosody model that incorporates the BERT model, the method also includes generating a corresponding prosodic syllable embedding for the syllable based on the wordpiece embedding associated with the word that includes the syllable and predicting a duration of the syllable by encoding linguistic features of each phoneme of the syllable with the corresponding prosodic syllable embedding for the syllable.