公开(公告)号：US20240232598A1

公开(公告)日：2024-07-11

申请号：US18469272

申请日：2023-09-18

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman ,  Bjarke Hammersholt Roune

IPC: G06N3/063 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367 ,  G06N3/045 ,  G06N3/086 ,  G06N3/10

CPC classification number: G06N3/063 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367 ,  G06N3/045 ,  G06N3/086 ,  G06N3/10

Abstract: Methods and systems, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform convolutional computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying the convolutional computation to be performed on a feature tensor and a filter and padding applied to the feature tensor prior to performing the convolutional computation; and generating instructions that when executed by the hardware circuit cause the hardware circuit to perform operations comprising: transferring feature tensor data from a main memory of the hardware circuit to a scratchpad memory of the hardware circuit; and repeatedly performing the following operations: identifying a current subset of the feature tensor; and determining whether a memory view into the scratchpad memory for the current subset is consistent with a memory view of the current subset in the main memory.

公开(公告)号：US11449739B2

公开(公告)日：2022-09-20

申请号：US16548555

申请日：2019-08-22

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman ,  Bjarke Hammersholt Roune

IPC: G06N3/063 ,  G06N3/08 ,  G06N3/04 ,  G06N3/10 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367

Abstract: Methods and systems, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform convolutional computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying the convolutional computation to be performed on a feature tensor and a filter and padding applied to the feature tensor prior to performing the convolutional computation; and generating instructions that when executed by the hardware circuit cause the hardware circuit to perform operations comprising: transferring feature tensor data from a main memory of the hardware circuit to a scratchpad memory of the hardware circuit; and repeatedly performing the following operations: identifying a current subset of the feature tensor; and determining whether a memory view into the scratchpad memory for the current subset is consistent with a memory view of the current subset in the main memory.

公开(公告)号：US20210049408A1

公开(公告)日：2021-02-18

申请号：US16543410

申请日：2019-08-16

Applicant: Google LLC

Inventor： Bjarke Hammersholt Roune ,  Sameer Kumar ,  Norman Paul Jouppi

IPC: G06K9/62 ,  G06N20/00 ,  H04L29/08

Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors for a network having one or more degraded nodes. A method comprises training a respective replica of a machine learning model on each node of multiple nodes organized in an n-dimensional network topology, combining the respective individual gradient vectors in the nodes to generate a final gradient vector by performing operations comprising: designating each group of nodes along the dimension as either a forwarding group or a critical group, updating, for each receiving node, a respective individual gradient vector with an intermediate gradient vector, performing a reduction on each critical group of nodes along the dimension to generate a respective partial final gradient vector for the critical group, and updating, for each critical group of nodes, an individual gradient vector for a representative node with the respective partial final gradient vector.

公开(公告)号：US20200042895A1

公开(公告)日：2020-02-06

申请号：US16495815

申请日：2018-02-08

Applicant: GOOGLE LLC

Inventor： Ian Moray Mclaren ,  Norman Paul Jouppi ,  Clifford Hsiang Chao ,  Gregory Michael Thorson ,  Bjarke Hammersholt Roune

IPC: G06N20/00 ,  G06F15/173

Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors and similarly structured data that are generated in parallel, for example, on nodes organized in a mesh or torus topology defined by connections in at least two dimension between the nodes. The methods provide parallel computation and communication between nodes in the topology.

公开(公告)号：US20220414441A1

公开(公告)日：2022-12-29

申请号：US17902776

申请日：2022-09-02

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman ,  Bjarke Hammersholt Roune

IPC: G06N3/063 ,  G06F17/16 ,  G06F17/15 ,  G06F30/20 ,  G06N3/04 ,  G06F30/27 ,  G06N3/08 ,  G06N3/10 ,  G06F30/367 ,  G06F30/18

Abstract: Methods and systems, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform convolutional computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying the convolutional computation to be performed on a feature tensor and a filter and padding applied to the feature tensor prior to performing the convolutional computation; and generating instructions that when executed by the hardware circuit cause the hardware circuit to perform operations comprising: transferring feature tensor data from a main memory of the hardware circuit to a scratchpad memory of the hardware circuit; and repeatedly performing the following operations: identifying a current subset of the feature tensor; and determining whether a memory view into the scratchpad memory for the current subset is consistent with a memory view of the current subset in the main memory.

公开(公告)号：US20210056396A1

公开(公告)日：2021-02-25

申请号：US16548555

申请日：2019-08-22

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman ,  Bjarke Hammersholt Roune

IPC: G06N3/063 ,  G06F17/16 ,  G06F17/15

Abstract: Methods and systems, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform convolutional computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying the convolutional computation to be performed on a feature tensor and a filter and padding applied to the feature tensor prior to performing the convolutional computation; and generating instructions that when executed by the hardware circuit cause the hardware circuit to perform operations comprising: transferring feature tensor data from a main memory of the hardware circuit to a scratchpad memory of the hardware circuit; and repeatedly performing the following operations: identifying a current subset of the feature tensor; and determining whether a memory view into the scratchpad memory for the current subset is consistent with a memory view of the current subset in the main memory.

公开(公告)号：US20180240039A1

公开(公告)日：2018-08-23

申请号：US15707104

申请日：2017-09-18

Applicant: Google LLC

Inventor： Ian Moray Mclaren ,  Norman Paul Jouppi ,  Clifford Hsiang Chao ,  Gregory Michael Thorson ,  Bjarke Hammersholt Roune

IPC: G06N99/00

CPC classification number: G06N99/005 ,  G06F15/17381

Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors and similarly structured data that are generated in parallel, for example, on nodes organized in a mesh or torus topology defined by connections in at least two dimension between the nodes. The methods provide parallel computation and communication between nodes in the topology.

公开(公告)号：US10055692B1

公开(公告)日：2018-08-21

申请号：US15707104

申请日：2017-09-18

Applicant: Google LLC

Inventor： Ian Moray Mclaren ,  Norman Paul Jouppi ,  Clifford Hsiang Chao ,  Gregory Michael Thorson ,  Bjarke Hammersholt Roune

IPC: G06N99/00

Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors and similarly structured data that are generated in parallel, for example, on nodes organized in a mesh or torus topology defined by connections in at least two dimension between the nodes. The methods provide parallel computation and communication between nodes in the topology.

公开(公告)号：US11763142B2

公开(公告)日：2023-09-19

申请号：US17902776

申请日：2022-09-02

Applicant: Google LLC

Inventor： David Alexander Majnemer ,  Blake Alan Hechtman ,  Bjarke Hammersholt Roune

IPC: G06N3/063 ,  G06N3/04 ,  G06N3/06 ,  G06N3/10 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367 ,  G06N3/086 ,  G06N3/045

CPC classification number: G06N3/063 ,  G06F17/15 ,  G06F17/16 ,  G06F30/18 ,  G06F30/20 ,  G06F30/27 ,  G06F30/367 ,  G06N3/045 ,  G06N3/086 ,  G06N3/10

Abstract: Methods and systems, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform convolutional computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying the convolutional computation to be performed on a feature tensor and a filter and padding applied to the feature tensor prior to performing the convolutional computation; and generating instructions that when executed by the hardware circuit cause the hardware circuit to perform operations comprising: transferring feature tensor data from a main memory of the hardware circuit to a scratchpad memory of the hardware circuit; and repeatedly performing the following operations: identifying a current subset of the feature tensor; and determining whether a memory view into the scratchpad memory for the current subset is consistent with a memory view of the current subset in the main memory.

公开(公告)号：US11715010B2

公开(公告)日：2023-08-01

申请号：US16543410

申请日：2019-08-16

Applicant: Google LLC

Inventor： Bjarke Hammersholt Roune ,  Sameer Kumar ,  Norman Paul Jouppi

IPC: G06N3/084 ,  G06N20/00 ,  G06F18/2115 ,  G06F18/23 ,  G06F18/214

CPC classification number: G06N3/084 ,  G06F18/2115 ,  G06F18/2148 ,  G06F18/23 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors for a network having one or more degraded nodes. A method comprises training a respective replica of a machine learning model on each node of multiple nodes organized in an n-dimensional network topology, combining the respective individual gradient vectors in the nodes to generate a final gradient vector by performing operations comprising: designating each group of nodes along the dimension as either a forwarding group or a critical group, updating, for each receiving node, a respective individual gradient vector with an intermediate gradient vector, performing a reduction on each critical group of nodes along the dimension to generate a respective partial final gradient vector for the critical group, and updating, for each critical group of nodes, an individual gradient vector for a representative node with the respective partial final gradient vector.