-
公开(公告)号:US11195521B2
公开(公告)日:2021-12-07
申请号:US16781273
申请日:2020-02-04
Applicant: Google LLC
Inventor: Navdeep Jaitly , Quoc V. Le , Oriol Vinyals , Samuel Bengio , Ilya Sutskever
IPC: G10L15/00 , G10L15/16 , G06N3/04 , G10L15/26 , G06F40/58 , G06F40/274 , G06F40/55 , G10L15/02 , G05B13/02
Abstract: A system can be configured to perform tasks such as converting recorded speech to a sequence of phonemes that represent the speech, converting an input sequence of graphemes into a target sequence of phonemes, translating an input sequence of words in one language into a corresponding sequence of words in another language, or predicting a target sequence of words that follow an input sequence of words in a language (e.g., a language model). In a speech recognizer, the RNN system may be used to convert speech to a target sequence of phonemes in real-time so that a transcription of the speech can be generated and presented to a user, even before the user has completed uttering the entire speech input.
-
公开(公告)号:US10977547B2
公开(公告)日:2021-04-13
申请号:US15349867
申请日:2016-11-11
Applicant: Google LLC
Inventor: Lukasz Mieczyslaw Kaiser , Ilya Sutskever
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing a convolutional gated recurrent neural network (CGRN). In one of the systems, the CGRN is configured to maintain a state that is a tensor having dimensions x by y by m, wherein x, y, and m are each greater than one, and for each of a plurality of time steps, update a currently maintained state by processing the currently maintained state through a plurality of convolutional gates.
-
公开(公告)号:US20210019604A1
公开(公告)日:2021-01-21
申请号:US16940131
申请日:2020-07-27
Applicant: Google LLC
Inventor: Gregory Sean Corrado , Ilya Sutskever , Jeffrey Adgate Dean
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.
-
公开(公告)号:US20200251099A1
公开(公告)日:2020-08-06
申请号:US16781273
申请日:2020-02-04
Applicant: Google LLC
Inventor: Navdeep Jaitly , Quoc V. Le , Oriol Vinyals , Samuel Bengio , Ilya Sutskever
Abstract: A system can be configured to perform tasks such as converting recorded speech to a sequence of phonemes that represent the speech, converting an input sequence of graphemes into a target sequence of phonemes, translating an input sequence of words in one language into a corresponding sequence of words in another language, or predicting a target sequence of words that follow an input sequence of words in a language (e.g., a language model). In a speech recognizer, the RNN system may be used to convert speech to a target sequence of phonemes in real-time so that a transcription of the speech can be generated and presented to a user, even before the user has completed uttering the entire speech input.
-
公开(公告)号:US10559300B2
公开(公告)日:2020-02-11
申请号:US16055414
申请日:2018-08-06
Applicant: Google LLC
Inventor: Navdeep Jaitly , Quoc V. Le , Oriol Vinyals , Samuel Bengio , Ilya Sutskever
Abstract: A system can be configured to perform tasks such as converting recorded speech to a sequence of phonemes that represent the speech, converting an input sequence of graphemes into a target sequence of phonemes, translating an input sequence of words in one language into a corresponding sequence of words in another language, or predicting a target sequence of words that follow an input sequence of words in a language (e.g., a language model). In a speech recognizer, the RNN system may be used to convert speech to a target sequence of phonemes in real-time so that a transcription of the speech can be generated and presented to a user, even before the user has completed uttering the entire speech input.
-
公开(公告)号:US20240176995A1
公开(公告)日:2024-05-30
申请号:US18466751
申请日:2023-09-13
Applicant: Google LLC
Inventor: Gregory Sean Corrado , Ilya Sutskever , Jeffrey Adgate Dean
CPC classification number: G06N3/047 , G06N3/042 , G06N3/044 , G06N3/063 , G16H50/20 , G06N3/02 , G06N3/08
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.
-
公开(公告)号:US11829882B2
公开(公告)日:2023-11-28
申请号:US17227010
申请日:2021-04-09
Applicant: Google LLC
Inventor: Geoffrey E. Hinton , Alexander Krizhevsky , Ilya Sutskever , Nitish Srivastava
Abstract: A system for training a neural network. A switch is linked to feature detectors in at least some of the layers of the neural network. For each training case, the switch randomly selectively disables each of the feature detectors in accordance with a preconfigured probability. The weights from each training case are then normalized for applying the neural network to test data.
-
公开(公告)号:US10977557B2
公开(公告)日:2021-04-13
申请号:US16523884
申请日:2019-07-26
Applicant: Google LLC
Inventor: Geoffrey E. Hinton , Alexander Krizhevsky , Ilya Sutskever , Nitish Srivastava
Abstract: A system for training a neural network. A switch is linked to feature detectors in at least some of the layers of the neural network. For each training case, the switch randomly selectively disables each of the feature detectors in accordance with a preconfigured probability. The weights from each training case are then normalized for applying the neural network to test data.
-
公开(公告)号:US10380482B2
公开(公告)日:2019-08-13
申请号:US14877071
申请日:2015-10-07
Applicant: Google LLC
Inventor: Ilya Sutskever , Wojciech Zaremba
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes obtaining partitioned training data for the neural network, wherein the partitioned training data comprises a plurality of training items each of which is assigned to a respective one of a plurality of partitions, wherein each partition is associated with a respective difficulty level; and training the neural network on each of the partitions in a sequence from a partition associated with an easiest difficulty level to a partition associated with a hardest difficulty level, wherein, for each of the partitions, training the neural network comprises: training the neural network on a sequence of training items that includes training items selected from the training items in the partition interspersed with training items selected from the training items in all of the partitions.
-
公开(公告)号:US10366329B2
公开(公告)日:2019-07-30
申请号:US15222870
申请日:2016-07-28
Applicant: Google LLC
Inventor: Geoffrey E. Hinton , Alexander Krizhevsky , Ilya Sutskever , Nitish Srivastava
Abstract: A system for training a neural network. A switch is linked to feature detectors in at least some of the layers of the neural network. For each training case, the switch randomly selectively disables each of the feature detectors in accordance with a preconfigured probability. The weights from each training case are then normalized for applying the neural network to test data.
-
-
-
-
-
-
-
-
-