公开(公告)号：US10181098B2

公开(公告)日：2019-01-15

申请号：US14731326

申请日：2015-06-04

Applicant: GOOGLE LLC

Inventor： Oriol Vinyals ,  Quoc V. Le ,  Ilya Sutskever

IPC: G06N3/02 ,  G06N3/04 ,  G06F17/28

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating representations of input sequences. One of the methods includes obtaining an input sequence, the input sequence comprising a plurality of inputs arranged according to an input order; processing the input sequence using a first long short term memory (LSTM) neural network to convert the input sequence into an alternative representation for the input sequence; and processing the alternative representation for the input sequence using a second LSTM neural network to generate a target sequence for the input sequence, the target sequence comprising a plurality of outputs arranged according to an output order.

公开(公告)号：US20240346298A1

公开(公告)日：2024-10-17

申请号：US18438368

申请日：2024-02-09

Applicant: Google LLC

Inventor： Alexander Krizhevsky ,  Ilya Sutskever ,  Geoffrey E. Hinton

IPC: G06N3/063 ,  G06F18/214 ,  G06N3/04 ,  G06N3/045 ,  G06N3/08 ,  G06T1/20 ,  G06V10/44

CPC classification number: G06N3/063 ,  G06F18/214 ,  G06N3/04 ,  G06N3/045 ,  G06N3/08 ,  G06T1/20 ,  G06V10/454

Abstract: A parallel convolutional neural network is provided. The CNN is implemented by a plurality of convolutional neural networks each on a respective processing node. Each CNN has a plurality of layers. A subset of the layers are interconnected between processing nodes such that activations are fed forward across nodes. The remaining subset is not so interconnected.

公开(公告)号：US20220284266A1

公开(公告)日：2022-09-08

申请号：US17704721

申请日：2022-03-25

Applicant: Google LLC

Inventor： Shixiang Gu ,  Timothy Paul Lillicrap ,  Ilya Sutskever ,  Sergey Vladimir Levine

IPC: G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for computing Q values for actions to be performed by an agent interacting with an environment from a continuous action space of actions. In one aspect, a system includes a value subnetwork configured to receive an observation characterizing a current state of the environment and process the observation to generate a value estimate; a policy subnetwork configured to receive the observation and process the observation to generate an ideal point in the continuous action space; and a subsystem configured to receive a particular point in the continuous action space representing a particular action; generate an advantage estimate for the particular action; and generate a Q value for the particular action that is an estimate of an expected return resulting from the agent performing the particular action when the environment is in the current state.

公开(公告)号：US11222252B2

公开(公告)日：2022-01-11

申请号：US16211635

申请日：2018-12-06

Applicant: Google LLC

Inventor： Oriol Vinyals ,  Quoc V. Le ,  Ilya Sutskever

IPC: G06N3/02 ,  G06N3/04 ,  G06F40/40

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating representations of input sequences. One of the methods includes obtaining an input sequence, the input sequence comprising a plurality of inputs arranged according to an input order; processing the input sequence using a first long short term memory (LSTM) neural network to convert the input sequence into an alternative representation for the input sequence; and processing the alternative representation for the input sequence using a second LSTM neural network to generate a target sequence for the input sequence, the target sequence comprising a plurality of outputs arranged according to an output order.

公开(公告)号：US20210224659A1

公开(公告)日：2021-07-22

申请号：US17227010

申请日：2021-04-09

Applicant: Google LLC

Inventor： Geoffrey E. Hinton ,  Alexander Krizhevsky ,  Ilya Sutskever ,  Nitish Srivastava

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/46

Abstract: A system for training a neural network. A switch is linked to feature detectors in at least some of the layers of the neural network. For each training case, the switch randomly selectively disables each of the feature detectors in accordance with a preconfigured probability. The weights from each training case are then normalized for applying the neural network to test data.

公开(公告)号：US10936828B2

公开(公告)日：2021-03-02

申请号：US16193387

申请日：2018-11-16

Applicant: Google LLC

Inventor： Quoc V. Le ,  Minh-Thang Luong ,  Ilya Sutskever ,  Oriol Vinyals ,  Wojciech Zaremba

IPC: G06F17/28 ,  G06F40/56 ,  G06N3/04 ,  G06F40/44 ,  G06F40/45 ,  G06F40/242 ,  G06F7/02 ,  G06F7/10 ,  G10L15/02 ,  G10L15/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for neural translation systems with rare word processing. One of the methods is a method training a neural network translation system to track the source in source sentences of unknown words in target sentences, in a source language and a target language, respectively and includes deriving alignment data from a parallel corpus, the alignment data identifying, in each pair of source and target language sentences in the parallel corpus, aligned source and target words; annotating the sentences in the parallel corpus according to the alignment data and a rare word model to generate a training dataset of paired source and target language sentences; and training a neural network translation model on the training dataset.

公开(公告)号：US20200327391A1

公开(公告)日：2020-10-15

申请号：US16859815

申请日：2020-04-27

Applicant: Google LLC

Inventor： Alexander Krizhevsky ,  Ilya Sutskever ,  Geoffrey E. Hinton

IPC: G06N3/04 ,  G06K9/66 ,  G06K9/62 ,  G06T1/20 ,  G06N3/063 ,  G06K9/46 ,  G06N3/08

Abstract: A parallel convolutional neural network is provided. The CNN is implemented by a plurality of convolutional neural networks each on a respective processing node. Each CNN has a plurality of layers. A subset of the layers are interconnected between processing nodes such that activations are fed forward across nodes. The remaining subset is not so interconnected.

公开(公告)号：US10656605B1

公开(公告)日：2020-05-19

申请号：US16401791

申请日：2019-05-02

Applicant: Google LLC

Inventor： Chung-Cheng Chiu ,  Navdeep Jaitly ,  Ilya Sutskever ,  Yuping Luo

IPC: G05B13/02 ,  G10L15/16 ,  G06N3/04 ,  G06F40/44

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a target sequence from a source sequence. In one aspect, the system includes a recurrent neural network configured to, at each time step, receive am input for the time step and process the input to generate a progress score and a set of output scores; and a subsystem configured to, at each time step, generate the recurrent neural network input and provide the input to the recurrent neural network; determine, from the progress score, whether or not to emit a new output at the time step; and, in response to determining to emit a new output, select an output using the output scores and emit the selected output as the output at a next position in the output order.

公开(公告)号：US10133739B2

公开(公告)日：2018-11-20

申请号：US14921925

申请日：2015-10-23

Applicant: GOOGLE LLC

Inventor： Quoc V. Le ,  Minh-Thang Luong ,  Ilya Sutskever ,  Oriol Vinyals ,  Wojciech Zaremba

IPC: G06F17/28 ,  G06F17/27 ,  G06F7/02 ,  G06F7/10 ,  G10L15/02 ,  G10L15/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for neural translation systems with rare word processing. One of the methods is a method training a neural network translation system to track the source in source sentences of unknown words in target sentences, in a source language and a target language, respectively and includes deriving alignment data from a parallel corpus, the alignment data identifying, in each pair of source and target language sentences in the parallel corpus, aligned source and target words; annotating the sentences in the parallel corpus according to the alignment data and a rare word model to generate a training dataset of paired source and target language sentences; and training a neural network translation model on the training dataset.

公开(公告)号：US12073307B2

公开(公告)日：2024-08-27

申请号：US18466751

申请日：2023-09-13

Applicant: Google LLC

Inventor： Gregory Sean Corrado ,  Ilya Sutskever ,  Jeffrey Adgate Dean

IPC: G06N3/047 ,  G06N3/042 ,  G06N3/044 ,  G06N3/063 ,  G16H50/20 ,  G06N3/02 ,  G06N3/08

CPC classification number: G06N3/047 ,  G06N3/042 ,  G06N3/044 ,  G06N3/063 ,  G16H50/20 ,  G06N3/02 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.