公开(公告)号：US11928577B2

公开(公告)日：2024-03-12

申请号：US16859815

申请日：2020-04-27

Applicant: Google LLC

Inventor： Alexander Krizhevsky ,  Ilya Sutskever ,  Geoffrey E. Hinton

IPC: G06N3/04 ,  G06F18/214 ,  G06N3/045 ,  G06N3/063 ,  G06N3/08 ,  G06T1/20 ,  G06V10/44

CPC classification number: G06N3/063 ,  G06F18/214 ,  G06N3/04 ,  G06N3/045 ,  G06N3/08 ,  G06T1/20 ,  G06V10/454

Abstract: A parallel convolutional neural network is provided. The CNN is implemented by a plurality of convolutional neural networks each on a respective processing node. Each CNN has a plurality of layers. A subset of the layers are interconnected between processing nodes such that activations are fed forward across nodes. The remaining subset is not so interconnected.

公开(公告)号：US11790216B2

公开(公告)日：2023-10-17

申请号：US16940131

申请日：2020-07-27

Applicant: Google LLC

Inventor： Gregory Sean Corrado ,  Ilya Sutskever ,  Jeffrey Adgate Dean

IPC: G06N3/08 ,  G06N3/047 ,  G16H50/20 ,  G06N3/042 ,  G06N3/044 ,  G06N3/063 ,  G06N3/02

CPC classification number: G06N3/047 ,  G06N3/042 ,  G06N3/044 ,  G06N3/063 ,  G16H50/20 ,  G06N3/02 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.

公开(公告)号：US20220101082A1

公开(公告)日：2022-03-31

申请号：US17643736

申请日：2021-12-10

Applicant: Google LLC

Inventor： Oriol Vinyals ,  Quoc V. Le ,  Ilya Sutskever

IPC: G06N3/02 ,  G06N3/04 ,  G06F40/40

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating representations of input sequences. One of the methods includes obtaining an input sequence, the input sequence comprising a plurality of inputs arranged according to an input order; processing the input sequence using a first long short term memory (LSTM) neural network to convert the input sequence into an alternative representation for the input sequence; and processing the alternative representation for the input sequence using a second LSTM neural network to generate a target sequence for the input sequence, the target sequence comprising a plurality of outputs arranged according to an output order.

公开(公告)号：US11288568B2

公开(公告)日：2022-03-29

申请号：US15429088

申请日：2017-02-09

Applicant: Google LLC

Inventor： Shixiang Gu ,  Timothy Paul Lillicrap ,  Ilya Sutskever ,  Sergey Vladimir Levine

IPC: G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for computing Q values for actions to be performed by an agent interacting with an environment from a continuous action space of actions. In one aspect, a system includes a value subnetwork configured to receive an observation characterizing a current state of the environment and process the observation to generate a value estimate; a policy subnetwork configured to receive the observation and process the observation to generate an ideal point in the continuous action space; and a subsystem configured to receive a particular point in the continuous action space representing a particular action; generate an advantage estimate for the particular action; and generate a Q value for the particular action that is an estimate of an expected return resulting from the agent performing the particular action when the environment is in the current state.

公开(公告)号：US10726327B2

公开(公告)日：2020-07-28

申请号：US15588535

申请日：2017-05-05

Applicant: Google LLC

Inventor： Gregory Sean Corrado ,  Ilya Sutskever ,  Jeffrey Adgate Dean

IPC: G06N3/08 ,  G06N3/04 ,  G16H50/20 ,  G06N3/063 ,  G06N3/02

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.

公开(公告)号：US20190180165A1

公开(公告)日：2019-06-13

申请号：US16211635

申请日：2018-12-06

Applicant: Google LLC

Inventor： Oriol Vinyals ,  Quoc V. Le ,  Ilya Sutskever

IPC: G06N3/02 ,  G06N3/04 ,  G06F17/28

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating representations of input sequences. One of the methods includes obtaining an input sequence, the input sequence comprising a plurality of inputs arranged according to an input order; processing the input sequence using a first long short term memory (LSTM) neural network to convert the input sequence into an alternative representation for the input sequence; and processing the alternative representation for the input sequence using a second LSTM neural network to generate a target sequence for the input sequence, the target sequence comprising a plurality of outputs arranged according to an output order.

公开(公告)号：US20180342238A1

公开(公告)日：2018-11-29

申请号：US16055414

申请日：2018-08-06

Applicant: Google LLC

Inventor： Navdeep Jaitly ,  Quoc V. Le ,  Oriol Vinyals ,  Samuel Bengio ,  Ilya Sutskever

IPC: G10L15/16 ,  G06N3/04 ,  G10L15/26 ,  G10L15/02 ,  G05B13/02 ,  G06F17/28 ,  G06F17/27

CPC classification number: G10L15/16 ,  G05B13/027 ,  G06F17/276 ,  G06F17/289 ,  G06N3/0445 ,  G10L15/02 ,  G10L15/26 ,  G10L2015/025

Abstract: A system can be configured to perform tasks such as converting recorded speech to a sequence of phonemes that represent the speech, converting an input sequence of graphemes into a target sequence of phonemes, translating an input sequence of words in one language into a corresponding sequence of words in another language, or predicting a target sequence of words that follow an input sequence of words in a language (e.g., a language model). In a speech recognizer, the RNN system may be used to convert speech to a target sequence of phonemes in real-time so that a transcription of the speech can be generated and presented to a user, even before the user has completed uttering the entire speech input.

公开(公告)号：US10043512B2

公开(公告)日：2018-08-07

申请号：US15349245

申请日：2016-11-11

Applicant: GOOGLE LLC

Inventor： Navdeep Jaitly ,  Quoc V. Le ,  Oriol Vinyals ,  Samuel Bengio ,  Ilya Sutskever

IPC: G06F15/00 ,  G10L15/16 ,  G06F17/28 ,  G10L15/02 ,  G05B13/02 ,  G06N3/04

Abstract: A system can be configured to perform tasks such as converting recorded speech to a sequence of phonemes that represent the speech, converting an input sequence of graphemes into a target sequence of phonemes, translating an input sequence of words in one language into a corresponding sequence of words in another language, or predicting a target sequence of words that follow an input sequence of words in a language (e.g., a language model). In a speech recognizer, the RNN system may be used to convert speech to a target sequence of phonemes in real-time so that a transcription of the speech can be generated and presented to a user, even before the user has completed uttering the entire speech input.

公开(公告)号：US10963779B2

公开(公告)日：2021-03-30

申请号：US15349955

申请日：2016-11-11

Applicant: Google LLC

Inventor： Quoc V. Le ,  Ilya Sutskever ,  Arvind Neelakantan

IPC: G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing operations using data from a data source. In one aspect, a method includes a neural network system including a controller neural network configured to: receive a controller input for a time step and process the controller input and a representation of a system input to generate: an operation score distribution that assigns a respective operation score to an operation and a data score distribution that assigns a respective data score in the data source. The neural network system can also include an operation subsystem configured to: perform operations to generate operation outputs, wherein at least one of the operations is performed on data in the data source, and combine the operation outputs in accordance with the operation score distribution and the data score distribution to generate a time step output for the time step.

公开(公告)号：US10657435B1

公开(公告)日：2020-05-19

申请号：US14877096

申请日：2015-10-07

Applicant: Google LLC

Inventor： Ilya Sutskever ,  Wojciech Zaremba

IPC: G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing an input sequence using a recurrent neural network to generate an output for the input sequence. One of the methods includes receiving the input sequence; generating a doubled sequence comprising a first instance of the input sequence followed by a second instance of the input sequence; and processing the doubled sequence using the recurrent neural network to generate the output for the input sequence.