公开(公告)号：US20240220799A1

公开(公告)日：2024-07-04

申请号：US18536074

申请日：2023-12-11

Applicant: Google LLC

Inventor： Kuan Fang ,  Alexander Toshkov Toshev

IPC: G06N3/08 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes receiving a current observation characterizing a current state of the environment as of the time step; generating an embedding of the current observation; processing scene memory data comprising embeddings of prior observations received at prior time steps using an encoder neural network, wherein the encoder neural network is configured to apply an encoder self-attention mechanism to the scene memory data to generate an encoded representation of the scene memory data; processing the encoded representation of the scene memory data and the embedding of the current observation using a decoder neural network to generate an action selection output; and causing the agent to perform the selected action.

公开(公告)号：US11455530B2

公开(公告)日：2022-09-27

申请号：US16602702

申请日：2019-11-20

Applicant: Google LLC

Inventor： Kuan Fang ,  Alexander Toshkov Toshev

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes receiving a current observation characterizing a current state of the environment as of the time step; generating an embedding of the current observation; processing scene memory data comprising embeddings of prior observations received at prior time steps using an encoder neural network, wherein the encoder neural network is configured to apply an encoder self-attention mechanism to the scene memory data to generate an encoded representation of the scene memory data; processing the encoded representation of the scene memory data and the embedding of the current observation using a decoder neural network to generate an action selection output; and causing the agent to perform the selected action.

公开(公告)号：US20210125038A1

公开(公告)日：2021-04-29

申请号：US17092837

申请日：2020-11-09

Applicant: Google LLC

Inventor： Samuel Bengio ,  Oriol Vinyals ,  Alexander Toshkov Toshev ,  Dumitru Erhan

IPC: G06N3/04 ,  G06F40/40

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating descriptions of input images. One of the methods includes obtaining an input image; processing the input image using a first neural network to generate an alternative representation for the input image; and processing the alternative representation for the input image using a second neural network to generate a sequence of a plurality of words in a target natural language that describes the input image.

公开(公告)号：US09940552B1

公开(公告)日：2018-04-10

申请号：US15069697

申请日：2016-03-14

Applicant: Google LLC

Inventor： Sergey Ioffe ,  Alexander Toshkov Toshev

IPC: G06K9/62

CPC classification number: G06K9/6276 ,  G06K9/6215 ,  G06K9/6267 ,  G06K9/628

Abstract: A linear function describing a framework for identifying an object of class k in an image sample x may be described by: wk*x+bk, where bk is the bias term. The higher the value obtained for a particular classifier, the better the match or strength of identity. A method is disclosed for classifier and/or content padding to convert dot-products to distances, applying a hashing and/or nearest neighbor technique on the resulting padded vectors, and preprocessing that may improve the hash entropy. A vector for an image, an audio, and/or a video may be received. One or more classifier vectors may be obtained. A padded image, video, and/or audio vector and classifier vector may be generated. A dot product may be approximated and a hashing and/or nearest neighbor technique may be performed on the approximated dot product to identify at least one class (or object) present in the image, video, and/or audio.

公开(公告)号：US11842277B2

公开(公告)日：2023-12-12

申请号：US17953222

申请日：2022-09-26

Applicant: Google LLC

Inventor： Kuan Fang ,  Alexander Toshkov Toshev

IPC: G06N3/08 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes receiving a current observation characterizing a current state of the environment as of the time step; generating an embedding of the current observation; processing scene memory data comprising embeddings of prior observations received at prior time steps using an encoder neural network, wherein the encoder neural network is configured to apply an encoder self-attention mechanism to the scene memory data to generate an encoded representation of the scene memory data; processing the encoded representation of the scene memory data and the embedding of the current observation using a decoder neural network to generate an action selection output; and causing the agent to perform the selected action.

公开(公告)号：US10832124B2

公开(公告)日：2020-11-10

申请号：US16538712

申请日：2019-08-12

Applicant: Google LLC

Inventor： Samy Bengio ,  Oriol Vinyals ,  Alexander Toshkov Toshev ,  Dumitru Erhan

IPC: G06N3/04 ,  G06F40/40

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating descriptions of input images. One of the methods includes obtaining an input image; processing the input image using a first neural network to generate an alternative representation for the input image; and processing the alternative representation for the input image using a second neural network to generate a sequence of a plurality of words in a target natural language that describes the input image.

公开(公告)号：US20180107658A1

公开(公告)日：2018-04-19

申请号：US15847580

申请日：2017-12-19

Applicant: Google LLC

Inventor： Gregory Don Hartrell ,  Debajit Ghosh ,  Matthew William Vaughan-Vail ,  John Michael Rivlin ,  Garth Conboy ,  Xinxing Gu ,  Alexander Toshkov Toshev

IPC: G06F17/28 ,  G06N3/02 ,  G06K9/72 ,  G06K9/00 ,  G06F17/21 ,  G06F17/24 ,  G06F17/22

CPC classification number: G06F17/2836 ,  G06F17/212 ,  G06F17/2229 ,  G06F17/243 ,  G06F17/289 ,  G06K9/00476 ,  G06K9/726 ,  G06N3/02 ,  G06N3/0454 ,  G06N3/084 ,  G06F17/00 ,  H04N7/00

Abstract: Digital graphic novel content is received and features of the graphic novel content are identified. At least one of the identified features includes text. Contextual information corresponding to the feature or features that include text is generated based on the identified features. The contextual information is used to aid translation of the text included in the feature or features that include text.

公开(公告)号：US12248875B2

公开(公告)日：2025-03-11

申请号：US18536074

申请日：2023-12-11

Applicant: Google LLC

Inventor： Kuan Fang ,  Alexander Toshkov Toshev

IPC: G06N3/08 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes receiving a current observation characterizing a current state of the environment as of the time step; generating an embedding of the current observation; processing scene memory data comprising embeddings of prior observations received at prior time steps using an encoder neural network, wherein the encoder neural network is configured to apply an encoder self-attention mechanism to the scene memory data to generate an encoded representation of the scene memory data; processing the encoded representation of the scene memory data and the embedding of the current observation using a decoder neural network to generate an action selection output; and causing the agent to perform the selected action.

公开(公告)号：US12014259B2

公开(公告)日：2024-06-18

申请号：US17092837

申请日：2020-11-09

Applicant: Google LLC

Inventor： Samy Bengio ,  Oriol Vinyals ,  Alexander Toshkov Toshev ,  Dumitru Erhan

IPC: G06N3/00 ,  G06F40/40 ,  G06N3/045 ,  G06N3/047

CPC classification number: G06N3/047 ,  G06F40/40 ,  G06N3/045

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating descriptions of input images. One of the methods includes obtaining an input image; processing the input image using a first neural network to generate an alternative representation for the input image; and processing the alternative representation for the input image using a second neural network to generate a sequence of a plurality of words in a target natural language that describes the input image.

公开(公告)号：US20230090658A1

公开(公告)日：2023-03-23

申请号：US17953222

申请日：2022-09-26

Applicant: Google LLC

Inventor： Kuan Fang ,  Alexander Toshkov Toshev

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes receiving a current observation characterizing a current state of the environment as of the time step; generating an embedding of the current observation; processing scene memory data comprising embeddings of prior observations received at prior time steps using an encoder neural network, wherein the encoder neural network is configured to apply an encoder self-attention mechanism to the scene memory data to generate an encoded representation of the scene memory data; processing the encoded representation of the scene memory data and the embedding of the current observation using a decoder neural network to generate an action selection output; and causing the agent to perform the selected action.