公开(公告)号：US20210158132A1

公开(公告)日：2021-05-27

申请号：US16698461

申请日：2019-11-27

Applicant: Amazon Technologies, Inc.

Inventor： Jeffrey T. Huynh ,  Ron Diamant ,  Hongbin Zheng ,  Yizhi Liu ,  Animesh Jain ,  Yida Wang ,  Vinod Sharma ,  Richard John Heaton ,  Randy Renfu Huang ,  Sundeep Amirineni ,  Drazen Borkovic

IPC: G06N3/063 ,  G06N3/04

Abstract: A computer-implemented method includes receiving a neural network model for implementation using a processing element array, where the neural network model includes a convolution operation on a set of input feature maps and a set of filters. The method also includes determining, based on the neural network model, that the convolution operation utilizes less than a threshold number of rows in the processing element array for applying a set of filter elements to the set of input feature maps, where the set of filter elements includes one filter element in each filter of the set of filters. The method further includes generating, for the convolution operation and based on the neural network model, a first instruction and a second instruction for execution by respective rows in the processing element array, where the first instruction and the second instruction use different filter elements of a filter in the set of filters.

公开(公告)号：US20200184245A1

公开(公告)日：2020-06-11

申请号：US16216485

申请日：2018-12-11

Applicant: Amazon Technologies, Inc.

Inventor： Randy Renfu Huang ,  Richard John Heaton ,  Andrea Olgiati ,  Ron Diamant

IPC: G06K9/03 ,  G06N3/04 ,  G06N3/08 ,  G06K9/62

Abstract: Systems and methods for performing improper input data detection are described. In one example, a system comprises: hardware circuits configured to receive input data and to perform computations of a neural network based on the input data to generate computation outputs; and an improper input detection circuit configured to: determine a relationship between the computation outputs of the hardware circuits and reference outputs; determine that the input data are improper based on the relationship; and perform an action based on determining that the input data are improper.

公开(公告)号：US12182688B2

公开(公告)日：2024-12-31

申请号：US16698236

申请日：2019-11-27

Applicant: Amazon Technologies, Inc.

Inventor： Animesh Jain ,  Yizhi Liu ,  Hongbin Zheng ,  Jeffrey T. Huynh ,  Haichen Li ,  Drazen Borkovic ,  Jindrich Zejda ,  Richard John Heaton ,  Randy Renfu Huang ,  Zhi Chen ,  Yida Wang

IPC: G06N3/063 ,  G06N3/04

Abstract: Methods and apparatuses for hierarchical partitioning of operators of a neural network for execution on an acceleration engine are provided. Neural networks are built in machine learning frameworks using neural network operators. The neural network operators are compiled into executable code for the acceleration engine. Development of new framework-level operators can exceed the capability to map the newly developed framework-level operators onto the acceleration engine. To enable neural networks to be executed on an acceleration engine, hierarchical partitioning can be used to partition the operators of the neural network. The hierarchical partitioning can identify operators that are supported by a compiler for execution on the acceleration engine, operators to be compiled for execution on a host processor, and operators to be executed on the machine learning framework.

公开(公告)号：US12130885B1

公开(公告)日：2024-10-29

申请号：US18052527

申请日：2022-11-03

Applicant: Amazon Technologies, Inc.

Inventor： Paul Gilbert Meyer ,  Thiam Khean Hah ,  Randy Renfu Huang ,  Ron Diamant ,  Vignesh Vivekraja

IPC: G06F17/16 ,  G06N3/04

CPC classification number: G06F17/16 ,  G06N3/04

Abstract: To take advantage of the architecture of a systolic array tailored to perform sparse matrix multiplications, a weight matrix can be converted into a set of constrained fine-grained sparse weight matrices. The conversion process may include receiving a request to perform a matrix multiplication operation with a weight matrix, and determining that the weight matrix satisfies a sparsity condition to convert the weight matrix into a set of constrained fine-grained sparse weight matrices. The weight matrix can then be converted into a set of constrained fine-grained sparse weight matrices. Computer instructions can then be generated for an integrated circuit device to perform the requested matrix multiplication operation as a set of sparse matrix multiplication operations using the set of constrained fine-grained sparse weight matrices.

公开(公告)号：US20240232630A1

公开(公告)日：2024-07-11

申请号：US18221454

申请日：2023-07-13

Applicant: Amazon Technologies, Inc.

Inventor： Vignesh Vivekraja ,  Thiam Khean Hah ,  Randy Renfu Huang ,  Ron Diamant ,  Richard John Heaton

IPC: G06N3/084 ,  G06N3/045 ,  G06N3/063 ,  G06N3/10

CPC classification number: G06N3/084 ,  G06N3/045 ,  G06N3/063 ,  G06N3/10

Abstract: Methods and systems for performing a training operation of a neural network are provided. In one example, a method comprises: performing backward propagation computations for a second layer of a neural network to generate second weight gradients; splitting the second weight gradients into portions; causing a hardware interface to exchange a first portion of the second weight gradients with the second computer system; performing backward propagation computations for a first layer of the neural network to generate first weight gradients when the exchange of the first portion of the second weight gradients is underway, the first layer being a lower layer than the second layer in the neural network; causing the hardware interface to transmit the first weight gradients to the second computer system; and causing the hardware interface to transmit the remaining portions of the second weight gradients to the second computer system.

公开(公告)号：US11941528B2

公开(公告)日：2024-03-26

申请号：US16588603

申请日：2019-09-30

Applicant: Amazon Technologies, Inc.

Inventor： Vignesh Vivekraja ,  Thiam Khean Hah ,  Randy Renfu Huang ,  Ron Diamant ,  Richard John Heaton

IPC: G06N3/084 ,  G06N3/063 ,  G06N3/045 ,  G06N3/10

CPC classification number: G06N3/084 ,  G06N3/045 ,  G06N3/063 ,  G06N3/10

Abstract: Methods and systems for performing a training operation of a neural network are provided. In one example, a method comprises: performing backward propagation computations for a second layer of a neural network to generate second weight gradients; splitting the second weight gradients into portions; causing a hardware interface to exchange a first portion of the second weight gradients with the second computer system; performing backward propagation computations for a first layer of the neural network to generate first weight gradients when the exchange of the first portion of the second weight gradients is underway, the first layer being a lower layer than the second layer in the neural network; causing the hardware interface to transmit the first weight gradients to the second computer system; and causing the hardware interface to transmit the remaining portions of the second weight gradients to the second computer system.

公开(公告)号：US20240020514A1

公开(公告)日：2024-01-18

申请号：US18143970

申请日：2023-05-05

Applicant: Amazon Technologies, Inc.

Inventor： Randy Renfu Huang ,  Richard John Heaton ,  Andrea Olgiati ,  Ron Diamant

IPC: G06N3/045 ,  G06N3/04 ,  G06N3/08 ,  G06F18/214

CPC classification number: G06N3/045 ,  G06N3/04 ,  G06N3/08 ,  G06F18/214

Abstract: Systems and methods for performing improper input data detection are described. In one example, a system comprises: hardware circuits configured to receive input data and to perform computations of a neural network based on the input data to generate computation outputs; and an improper input detection circuit configured to: determine a relationship between the computation outputs of the hardware circuits and reference outputs; determine that the input data are improper based on the relationship; and perform an action based on determining that the input data are improper.

公开(公告)号：US11868875B1

公开(公告)日：2024-01-09

申请号：US16127170

申请日：2018-09-10

Applicant: Amazon Technologies, Inc.

Inventor： Ron Diamant ,  Randy Renfu Huang ,  Jeffrey T. Huynh ,  Sundeep Amirineni

IPC: G06N3/065 ,  G11C11/54 ,  G06N3/049

CPC classification number: G06N3/065 ,  G06N3/049 ,  G11C11/54

Abstract: Provided are systems and methods for operating a neural network processor, wherein the processor includes an input selector circuit that can be configured to select the data that will be input into the processor's computational array. In various implementations, the selector circuit can determine, for a row of the array, whether the row input will be the output from a buffer memory or data that the input selector circuit has selected for a different row. The row can receive an input feature map from a set of input data or an input feature map that was selected for inputting into a different row, such that the input feature map is input into more than one row at a time. The selector circuit can also include a delay circuit, so that the duplicated input feature map can be input into the computational array later than the original input feature map.

公开(公告)号：US11809981B1

公开(公告)日：2023-11-07

申请号：US16698753

申请日：2019-11-27

Applicant: Amazon Technologies, Inc.

Inventor： Animesh Jain ,  Tobias Joseph Kastulus Edler von Koch ,  Yizhi Liu ,  Taemin Kim ,  Jindrich Zejda ,  Yida Wang ,  Vinod Sharma ,  Richard John Heaton ,  Randy Renfu Huang

IPC: G06N3/063 ,  G06F9/30 ,  G06F9/54

CPC classification number: G06N3/063 ,  G06F9/30007 ,  G06F9/545

Abstract: A method of generating executable instructions for a computing system is provided. The method comprises: receiving a first set of instructions including a kernel of a first operator and a kernel of a second operator, the kernel of the first operator including instructions of the first operator and write instructions to a virtual data node, the kernel of the second operator including instructions of the second operator and read instructions to the virtual data node; determining, based on a mapping between the write instructions and read instructions, instructions of data transfer operations between the first operator and the second operator; and generating a second set of instructions representing a fused operator of the first operator and the second operator, the second set of instructions including the instructions of the first operator, the instructions of the second operator, and the instructions of the data transfer operations.

公开(公告)号：US11803736B1

公开(公告)日：2023-10-31

申请号：US16917015

申请日：2020-06-30

Applicant: Amazon Technologies, Inc.

Inventor： Paul Gilbert Meyer ,  Thiam Khean Hah ,  Randy Renfu Huang ,  Ron Diamant ,  Vignesh Vivekraja

IPC: G06N3/063 ,  G06F17/16 ,  G06F7/544 ,  G06F9/38

CPC classification number: G06N3/063 ,  G06F7/5443 ,  G06F9/3893 ,  G06F17/16 ,  G06F2207/4824

Abstract: A systolic array can implement an architecture tailored to perform matrix multiplications on constrained fine-grained sparse weight matrices. Each processing element in the systolic array may include a weight register configured to store a weight value, and a multiplexor configured to select a feature map (FMAP) input element from multiple FMAP input data buses based on metadata associated with the weight value. Each processing element may also include a multiplier configured to multiply the selected feature map input element with the weight value to generate a multiplication result, and an adder configured to add the multiplication result to a partial sum input to generate a partial sum output.