公开(公告)号：US11308027B1

公开(公告)日：2022-04-19

申请号：US16915795

申请日：2020-06-29

Applicant: Amazon Technologies, Inc.

Inventor： Thomas A Volpe ,  Sundeep Amirineni ,  Thomas Elmer

IPC: G06F15/80 ,  G06F7/53 ,  G06F7/544 ,  G06F7/505 ,  G06F9/30 ,  G06F9/38

Abstract: Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each column of the systolic array can include multiple busses enabling independent transmission of input partial sums along the respective bus. Each processing element of a given columnar bus can receive an input partial sum from a prior element of the given columnar bus, and perform arithmetic operations on the input partial sum. Each processing element can generate an output partial sum based on the arithmetic operations, provide the output partial sum to a next processing element of the given columnar bus, without the output partial sum being processed by a processing element of the column located between the two processing elements that uses a different columnar bus. Use of columnar busses can enable parallelization to increase speed or enable increased latency at individual processing elements.

公开(公告)号：US12182064B2

公开(公告)日：2024-12-31

申请号：US18446357

申请日：2023-08-08

Applicant: Amazon Technologies, Inc.

Inventor： Thomas A Volpe ,  Sundeep Amirineni ,  Thomas Elmer

IPC: G06F15/80 ,  G06F7/505 ,  G06F7/53 ,  G06F7/544 ,  G06F9/30 ,  G06F9/38

Abstract: Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each column of the systolic array can include multiple busses enabling independent transmission of input partial sums along the respective bus. Each processing element of a given columnar bus can receive an input partial sum from a prior element of the given columnar bus, and perform arithmetic operations on the input partial sum. Each processing element can generate an output partial sum based on the arithmetic operations, provide the output partial sum to a next processing element of the given columnar bus, without the output partial sum being processed by a processing element of the column located between the two processing elements that uses a different columnar bus. Use of columnar busses can enable parallelization to increase speed or enable increased latency at individual processing elements.

公开(公告)号：US11520731B1

公开(公告)日：2022-12-06

申请号：US17091964

申请日：2020-11-06

Applicant: Amazon Technologies, Inc.

Inventor： Ron Diamant ,  Thomas A Volpe

IPC: G06F15/80 ,  G06F9/30 ,  G06N20/00 ,  G06F15/78 ,  G06F15/17 ,  G06F13/38 ,  G06F13/366 ,  G06F1/3206 ,  G06F15/76

Abstract: Throttling recommendations for a systolic array may be arbitrated. Throttling recommendations may be received at an arbiter for a systolic array from different sources, such as one or more monitors implemented in an integrated circuit along with the systolic array or sources external to the integrated circuit with the systolic array. A strongest throttling recommendation may be selected. The rate at which data enters the systolic array may be modified according to the strongest throttling recommendation.

公开(公告)号：US11442890B1

公开(公告)日：2022-09-13

申请号：US17091979

申请日：2020-11-06

Applicant: Amazon Technologies, Inc.

Inventor： Thomas A Volpe

IPC: G06F15/80 ,  G06F9/30 ,  G06F11/34 ,  G06F15/78 ,  G06N20/00 ,  G06F13/366 ,  G06F15/76

Abstract: On-circuit data activity monitoring may be performed for a systolic array. A current data activity measurement may be determined for changes in input data for processing at a systolic array and compared with a prior data activity measurement. Based on the comparison, a throttling recommendation may be provided to a management component to determine whether to perform the throttling recommendation.

公开(公告)号：US11232062B1

公开(公告)日：2022-01-25

申请号：US16915783

申请日：2020-06-29

Applicant: Amazon Technologies, Inc.

Inventor： Thomas A Volpe ,  Sundeep Amirineni ,  Thomas Elmer

IPC: G06F15/80 ,  G06F13/40 ,  G06F7/544

Abstract: Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each column of the systolic array can include multiple busses enabling independent transmission of input partial sums along the respective bus. Each processing element can include a plurality of interconnects to receive a plurality of inputs corresponding to the multiple busses. Each processing element of a given columnar bus can receive an input from a prior element of the given columnar bus at an active bus position and perform arithmetic operations on the input. Each processing element can further receive a plurality of inputs at passive bus positions and provide the plurality of inputs to subsequent processing elements without the plurality of inputs being processed by the processing element. Use of columnar busses can enable parallelization to increase speed or enable increased latency at individual processing elements.

公开(公告)号：US11113233B1

公开(公告)日：2021-09-07

申请号：US16915828

申请日：2020-06-29

Applicant: Amazon Technologies, Inc.

Inventor： Thomas A Volpe

IPC: G06F15/80 ,  G06F9/38 ,  G06F9/30

Abstract: Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each row of the systolic array can include multiple busses enabling independent transmission of inputs along the respective bus. Each processing element of a given row-oriented bus can receive an input from a prior element of the given row-oriented bus, and perform arithmetic operations on the input. The systolic array can be divided into a plurality of sub-arrays corresponding to a row-oriented bus where each sub-array is separated by a shifter. Each shifter can shift a row-oriented bus into the active bus position for a given sub-array. Use of row-oriented busses can enable parallelization to increase speed or enable increased latency at individual processing elements.