公开(公告)号：US12205018B2

公开(公告)日：2025-01-21

申请号：US18329418

申请日：2023-06-05

Applicant: Google LLC

Inventor： Reginald Clifford Young ,  Geoffrey Irving

IPC: G06N3/063 ,  G06F7/78 ,  G06F17/16 ,  G06N3/04 ,  G06N3/045 ,  G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying a transpose operation to be performed on a first neural network matrix; and generating instructions that when executed by the hardware circuit cause the hardware circuit to transpose the first neural network matrix by performing first operations, wherein the first operations include repeatedly performing the following second operations: for a current subdivision of the first neural network matrix that divides the first neural network matrix into one or more current submatrices, updating the first neural network matrix by swapping an upper right quadrant and a lower left quadrant of each current submatrix, and subdividing each current submatrix into respective new submatrices to update the current subdivision.

公开(公告)号：US20240152740A1

公开(公告)日：2024-05-09

申请号：US18329418

申请日：2023-06-05

Applicant: Google LLC

Inventor： Reginald Clifford Young ,  Geoffrey Irving

IPC: G06N3/063 ,  G06F7/78 ,  G06F17/16 ,  G06N3/04

CPC classification number: G06N3/063 ,  G06F7/78 ,  G06F17/16 ,  G06N3/04 ,  G06F2207/4824 ,  G06N3/084

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium. In one aspect, a method includes the actions of receiving a request to perform computations for a neural network on a hardware circuit having a matrix computation unit, the request specifying a transpose operation to be performed on a first neural network matrix; and generating instructions that when executed by the hardware circuit cause the hardware circuit to transpose the first neural network matrix by performing first operations, wherein the first operations include repeatedly performing the following second operations: for a current subdivision of the first neural network matrix that divides the first neural network matrix into one or more current submatrices, updating the first neural network matrix by swapping an upper right quadrant and a lower left quadrant of each current submatrix, and subdividing each current submatrix into respective new submatrices to update the current subdivision.

公开(公告)号：US11829321B2

公开(公告)日：2023-11-28

申请号：US17703479

申请日：2022-03-24

Applicant: Google LLC

Inventor： Reginald Clifford Young ,  Trevor Gale ,  Sushma Honnavara-Prasad ,  Paolo Mantovani

IPC: G06F15/80

CPC classification number: G06F15/8046 ,  G06F15/8069 ,  G06F15/8084

Abstract: A systolic array cell is described, the cell including two general-purpose arithmetic logic units (ALUs) and register-file. A plurality of the cells may be configured in a matrix or array, such that the output of the first ALU in a first cell is provided to a second cell to the right of the first cell, and the output of the second ALU in the first cell is provided to a third cell below the first cell. The two ALUs in each cell of the array allow for processing of a different instruction in each cycle.

公开(公告)号：US20230325347A1

公开(公告)日：2023-10-12

申请号：US17703479

申请日：2022-03-24

Applicant: Google LLC

Inventor： Reginald Clifford Young ,  Trevor Gale ,  Sushma Honnavara-Prasad ,  Paolo Mantovani

IPC: G06F15/80

CPC classification number: G06F15/8046 ,  G06F15/8069 ,  G06F15/8084

Abstract: A systolic array cell is described, the cell including two general-purpose arithmetic logic units (ALUs) and register-file. A plurality of the cells may be configured in a matrix or array, such that the output of the first ALU in a first cell is provided to a second cell to the right of the first cell, and the output of the second ALU in the first cell is provided to a third cell below the first cell. The two ALUs in each cell of the array allow for processing of a different instruction in each cycle.

公开(公告)号：US20230041163A1

公开(公告)日：2023-02-09

申请号：US17791771

申请日：2021-01-15

Applicant: Google LLC

Inventor： Erich Konrad Elsen ,  Trevor John Gale ,  Reginald Clifford Young

IPC: G06N3/04 ,  G06F9/38

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for parallelizing matrix operations. One of the methods includes implementing a neural network on a parallel processing device, the neural network comprising at least one sparse neural network layer, the sparse neural network layer being configured to receive an input matrix and perform matrix multiplication between the input matrix and a sparse weight matrix to generate an output matrix, the method comprising: for each row of the M rows of the output matrix, determining a plurality of tiles that each include one or more elements from the row; assigning, for each tile of each row, the tile to a respective one of a plurality of thread blocks of the parallel processing device; and computing, for each tile, respective values for each element in the tile using the respective thread block to which the tile was assigned.

公开(公告)号：US11216726B2

公开(公告)日：2022-01-04

申请号：US16139258

申请日：2018-09-24

Applicant: Google LLC

Inventor： Reginald Clifford Young

IPC: G06N3/08 ,  G06N3/063 ,  G06N5/04 ,  G06N3/06

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a respective neural network output for each of a plurality of inputs, the method comprising, for each of the neural network layers: receiving a plurality of inputs to be processed at the neural network layer; forming one or more batches of inputs from the plurality of inputs, each batch having a number of inputs up to the respective batch size for the neural network layer; selecting a number of the one or more batches of inputs to process, where a count of the inputs in the number of the one or more batches is greater than or equal to the respective associated batch size of a subsequent layer in the sequence; and processing the number of the one or more batches of inputs to generate the respective neural network layer output.

公开(公告)号：US11049016B2

公开(公告)日：2021-06-29

申请号：US16824411

申请日：2020-03-19

Applicant: Google LLC

Inventor： Jonathan Ross ,  Norman Paul Jouppi ,  Andrew Everett Phelps ,  Reginald Clifford Young ,  Thomas Norrie ,  Gregory Michael Thorson ,  Dan Luu

IPC: G06N3/08 ,  G06F15/80 ,  G06N3/063 ,  G06N5/04

Abstract: A circuit for performing neural network computations for a neural network comprising a plurality of neural network layers, the circuit comprising: a matrix computation unit configured to, for each of the plurality of neural network layers: receive a plurality of weight inputs and a plurality of activation inputs for the neural network layer, and generate a plurality of accumulated values based on the plurality of weight inputs and the plurality of activation inputs; and a vector computation unit communicatively coupled to the matrix computation unit and configured to, for each of the plurality of neural network layers: apply an activation function to each accumulated value generated by the matrix computation unit to generate a plurality of activated values for the neural network layer.

公开(公告)号：US11023804B2

公开(公告)日：2021-06-01

申请号：US16017708

申请日：2018-06-25

Applicant: Google LLC

Inventor： Reginald Clifford Young

IPC: G06N3/04 ,  G06N3/063 ,  G06N7/00

Abstract: Systems, methods, and apparatus, including computer programs encoded on a computer storage medium for processing a network input through a neural network having one or more initial neural network layers followed by a softmax output layer. In one aspect, the methods include obtaining a layer output generated by the one or more initial neural network layers and processing the layer output through the softmax output layer to generate a neural network output. Processing the layer output through the softmax output layer includes determining, for each possible output value, a number of occurrences in the layer output values; for each possible output value occurring in the layer output values, determining a respective exponentiation measure; determining a normalization factor for the layer output by combining the exponentiation measures in accordance with the number of occurrences of the possible output values; and determining, for each of layer output values, a softmax probability value.

公开(公告)号：US10733505B2

公开(公告)日：2020-08-04

申请号：US15348199

申请日：2016-11-10

Applicant: Google LLC

Inventor： Reginald Clifford Young ,  William John Gulland

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063 ,  G06N3/02

Abstract: Methods for receiving a request to process, on a hardware circuit, a neural network comprising a first convolutional neural network layer having a stride greater than one, and in response, generating instructions that cause the hardware circuit to, during processing of an input tensor, generate a layer output tensor equivalent to an output of the first convolutional neural network layer by processing the input tensor using a second convolutional neural network layer having a stride equal to one but that is otherwise equivalent to the first convolutional neural network layer to generate a first tensor, zeroing out elements of the first tensor that would not have been generated if the second convolutional neural network layer had the stride of the first convolutional neural network layer to generate a second tensor, and performing max pooling on the second tensor to generate the layer output tensor.

公开(公告)号：US10037490B2

公开(公告)日：2018-07-31

申请号：US15467294

申请日：2017-03-23

Applicant: Google LLC

Inventor： Reginald Clifford Young ,  William John Gulland

IPC: G06N3/08 ,  G06N5/04

Abstract: Methods and systems for receiving a request to implement a neural network comprising an average pooling layer on a hardware circuit, and in response, generating instructions that when executed by the hardware circuit, cause the hardware circuit to, during processing of a network input by the neural network, generate a layer output tensor that is equivalent to an output of the average pooling neural network layer by performing a convolution of an input tensor to the average pooling neural network layer and a kernel with a size equal to a window of the average pooling neural network layer and composed of elements that are each an identity matrix to generate a first tensor, and performing operations to cause each element of the first tensor to be divided by a number of elements in the window of the average pooling neural network layer to generate an initial output tensor.