公开(公告)号：US20220108215A1

公开(公告)日：2022-04-07

申请号：US17423601

申请日：2019-12-16

Applicant: Google LLC

Inventor： Krzysztof Choromanski ,  Vikas Sindhwani ,  Aldo Pacchiano Camacho

IPC: G06N20/00 ,  G06F17/11

Abstract: The present disclosure provides iterative blackbox optimization techniques that estimate the gradient of a function. According to an aspect of the present disclosure, a plurality of perturbations used at each iteration can be sampled from a non-orthogonal sampling distribution. As one example, in some implementations, perturbations that have been previously evaluated in previous iterations can be re-used at the current iteration. thereby conserving computing resources because the re-used perturbations do not need to be re-evaluated at the current iteration. In another example, in addition or alternatively to the use of previously evaluated perturbations, the perturbations evaluated at the current iteration can be sampled from a non-orthogonal sampling distribution.

公开(公告)号：US20230130634A1

公开(公告)日：2023-04-27

申请号：US17936547

申请日：2022-09-29

Applicant: Google LLC

Inventor： Tara N. Sainath ,  Rami Botros ,  Anmol Gulati ,  Krzysztof Choromanski ,  Ruoming Pang ,  Trevor Strohman ,  Weiran Wang ,  Jiahui Yu

IPC: G10L15/16 ,  G10L15/22 ,  G10L15/06

Abstract: A computer-implemented method includes receiving a sequence of acoustic frames as input to an automatic speech recognition (ASR) model. Here, the ASR model includes a causal encoder and a decoder. The method also includes generating, by the causal encoder, a first higher order feature representation for a corresponding acoustic frame in the sequence of acoustic frames. The method also includes generating, by the decoder, a first probability distribution over possible speech recognition hypotheses. Here, the causal encoder includes a stack of causal encoder layers each including a Recurrent Neural Network (RNN) Attention-Performer module that applies linear attention.

公开(公告)号：US12190869B2

公开(公告)日：2025-01-07

申请号：US17936547

申请日：2022-09-29

Applicant: Google LLC

Inventor： Tara N. Sainath ,  Rami Botros ,  Anmol Gulati ,  Krzysztof Choromanski ,  Ruoming Pang ,  Trevor Strohman ,  Weiran Wang ,  Jiahui Yu

IPC: G10L15/16 ,  G10L15/06 ,  G10L15/22

Abstract: A computer-implemented method includes receiving a sequence of acoustic frames as input to an automatic speech recognition (ASR) model. Here, the ASR model includes a causal encoder and a decoder. The method also includes generating, by the causal encoder, a first higher order feature representation for a corresponding acoustic frame in the sequence of acoustic frames. The method also includes generating, by the decoder, a first probability distribution over possible speech recognition hypotheses. Here, the causal encoder includes a stack of causal encoder layers each including a Recurrent Neural Network (RNN) Attention-Performer module that applies linear attention.