公开(公告)号：US12050997B2

公开(公告)日：2024-07-30

申请号：US16884130

申请日：2020-05-27

申请人： INTERNATIONAL BUSINESS MACHINES CORPORATION

发明人： Hsinyu Tsai ,  Geoffrey Burr ,  Pritish Narayanan ,  Malte Johannes Rasch

IPC分类号： G06N3/084 ,  G11C7/10 ,  G11C11/54 ,  G11C13/00 ,  G06N3/063

CPC分类号： G06N3/084 ,  G11C7/1006 ,  G11C11/54 ,  G11C13/0069 ,  G06N3/063 ,  G11C2213/77 ,  G11C2213/79

摘要： A computer implemented method for implementing a convolutional neural network (CNN) using a crosspoint array includes configuring the crosspoint array to implement a convolution layer by storing one or more weights in crosspoint devices of the array. The method further includes making multiple copies of the weights and training the CNN. Training the CNN includes mapping input data of the convolution layer to the crosspoint array in a row-by-row manner. Further the excitation is input in a row-by-row manner into the crosspoint array, thereby creating row-by-row forward output from the crosspoint array. Further, outputs from the crosspoint devices are stored to corresponding integrators. Errors in the outputs as compared to a desired output, from multiple rows are computed and back propagated in a row-by-row manner into the crosspoint array, the computed errors transmitted to a previous convolution layer.

公开(公告)号：US20220391684A1

公开(公告)日：2022-12-08

申请号：US17336392

申请日：2021-06-02

申请人： International Business Machines Corporation

发明人： Malte Johannes Rasch

IPC分类号： G06N3/08 ,  G06N3/063 ,  G06F15/80

摘要： A computer-implemented method, computer program product, and/or computer system that performs the following operations: (i) receiving outputs pertaining to a first step of a training process being performed on an analog resistive processing unit (RPU) array, the analog RPU array corresponding to a layer of a deep neural network (DNN); (ii) converting the outputs into a format having less precision, yielding converted outputs; (iii) initiating a calculation of an update parameter for a first step update pass of the layer utilizing the converted outputs; and (v) based, at least in part, on receiving outputs pertaining to a second step of the training process being performed on the analog RPU array, applying the update parameter for the first step update pass of the layer to the analog RPU array.

公开(公告)号：US20220138579A1

公开(公告)日：2022-05-05

申请号：US17086856

申请日：2020-11-02

申请人： INTERNATIONAL BUSINESS MACHINES CORPORATION

发明人： Malte Johannes Rasch ,  Tayfun Gokmen

IPC分类号： G06N3/08 ,  G06F16/22 ,  G06N3/04

摘要： A method is presented for artificial neural network training. The method includes storing weight values in an array of resistive processing unit (RPU) devices, wherein the array of RPU devices represents a weight matrix, defining the weight matrix to have an output dimension that is smaller than the input dimension such that the weight matrix has a rectangular configuration, and converting the weight matrix from a rectangular configuration to a more square-shaped configuration by repeating or concatenating the rectangular configuration of the weight matrix to increase a signal strength of a backward pass signal by copying an input of repeated weight elements during a forward cycle pass, summing output computations from the repeated weight elements, updating each of the repeated weight elements according to a backpropagated error or alternatively updating only one of the repeated weight elements by setting all forward values except one to zero during an update pass.

公开(公告)号：US20240232610A9

公开(公告)日：2024-07-11

申请号：US18048436

申请日：2022-10-20

申请人： INTERNATIONAL BUSINESS MACHINES CORPORATION

发明人： Malte Johannes Rasch

IPC分类号： G06N3/08 ,  G06N3/047

CPC分类号： G06N3/08 ,  G06N3/047

摘要： A computer implemented method includes performing a gradient update for a stochastic gradient descent (SGD) of a deep neural network (DNN) using a first set of hidden weights stored in a first matrix comprising a Resistive Processing Unit (RPU) crossbar array. A second matrix comprising a second set of hidden weights is stored in a digital medium. A third matrix comprising a set of reference values is computed upon a transfer cycle of the first set of weights from the first matrix to the second matrix, accounting for a sign-change (chopper). The third matrix is stored in the digital medium. A third set of weights is updated for the DNN from the second matrix when a threshold is reached for the second set of weights, in a fourth matrix comprising a RPU crossbar array.

公开(公告)号：US20240202512A1

公开(公告)日：2024-06-20

申请号：US18083673

申请日：2022-12-19

申请人： International Business Machines Corporation

发明人： Nanbo Gong ,  Takashi Ando ,  Guy M. Cohen ,  Malte Johannes Rasch

IPC分类号： G06N3/065 ,  G06F12/02 ,  G06N3/0985

CPC分类号： G06N3/065 ,  G06F12/0238 ,  G06N3/0985 ,  G06F2212/202

摘要： Analog memory-based activation function for an artificial neural network can be provided. An apparatus can include at least two non-volatile memory devices connected in parallel such that the current can flow through one of the two non-volatile memory devices depending on the voltage level driving the current. To control which branch an input current flows through, each of the two non-volatile memory devices can be connected to a circuit element that can function as a switch, for example, a diode such as a semiconductor diode, a transistor, or another circuit element. Such apparatus can implement an analog memory-based activation function, for example, for an analog memory-based artificial neural network.

公开(公告)号：US20220207376A1

公开(公告)日：2022-06-30

申请号：US17134814

申请日：2020-12-28

申请人： International Business Machines Corporation

发明人： Tayfun Gokmen ,  Oguzhan Murat Onen ,  Chai Wah Wu ,  Mark S. Squillante ,  Malte Johannes Rasch ,  Tomasz J. Nowicki ,  Wilfried Haensch ,  Lior Horesh ,  Vasileios Kalantzis ,  Vanessa Lopez-Marrero

IPC分类号： G06N3/08 ,  G06N3/063 ,  G06N3/04 ,  G06F17/16

摘要： Matrix inversion systems and methods are implemented using an analog resistive processing unit (RPU) array for hardware accelerated computing. A request is received from an application to compute an inverse matrix of a given matrix, and a matrix inversion process is performed in response to the received request. The matrix inversion process includes storing a first estimated inverse matrix of the given matrix in an array RPU cells, performing a first iterative process on the first estimated inverse matrix stored in the array of RPU cells to converge the first estimated inverse matrix to a second estimated inverse matrix of the given matrix, and reading the second estimated inverse matrix from the array of RPU cells upon completion of the first iterative process. An inverse matrix is returned to the application, wherein the returned inverse matrix is based, at least in part, on the second estimated inverse matrix.

公开(公告)号：US20220414445A1

公开(公告)日：2022-12-29

申请号：US17362684

申请日：2021-06-29

申请人： International Business Machines Corporation

发明人： Malte Johannes Rasch ,  Mattia Rigotti

IPC分类号： G06N3/063 ,  G06N3/04 ,  G06N3/08

摘要： A neural network includes a plurality of analog arrays comprise all synaptic weights of the neural network. The neural network also includes digital modules that are co-trained along with the plurality of analog arrays. The digital modules are intermittently connected and intermittently activated when the neural network is in production. When activated and connected, the digital modules may correct weights of the analog arrays.

公开(公告)号：US11443171B2

公开(公告)日：2022-09-13

申请号：US16929168

申请日：2020-07-15

申请人： International Business Machines Corporation

发明人： Seyoung Kim ,  Oguzhan Murat Onen ,  Tayfun Gokmen ,  Malte Johannes Rasch

IPC分类号： G06N3/04 ,  G06N3/063 ,  G06N3/08 ,  G06F9/30

摘要： Provided are embodiments for a computer-implemented method, a system, and a computer program product for updating an analog crossbar array. Embodiment include receiving a number used in matrix multiplication to represent using pulse generation for a crossbar array, and receiving a bit-length to represent the number. Embodiments also include selecting pulse positions in a pulse sequence having the bit length to represent the number, performing a computation using the selected pulse positions in the pulse sequence, and updating the crossbar array using the computation.

公开(公告)号：US20220197639A1

公开(公告)日：2022-06-23

申请号：US17131034

申请日：2020-12-22

申请人： INTERNATIONAL BUSINESS MACHINES CORPORATION ,  Ramot at Tel Aviv University Ltd.

发明人： Malte Johannes Rasch ,  Oguzhan Murat Onen ,  Tayfun Gokmen ,  Chai Wah Wu ,  Mark S. Squillante ,  Tomasz J. Nowicki ,  Wilfried Haensch ,  Lior Horesh ,  Vasileios Kalantzis ,  Haim Avron

IPC分类号： G06F9/30 ,  G06F9/50

摘要： Methods and systems for solving a linear system include setting resistances in an array of settable electrical resistances in accordance with values of an input matrix. A series of input vectors is applied to the array as voltages to generate a series of respective output vectors. Each input vector in the series of vectors is updated based on comparison of the respective output vectors to a target vector. A solution of a linear system is determined that includes the input matrix based on the updated input vectors.

公开(公告)号：US11366876B2

公开(公告)日：2022-06-21

申请号：US16910975

申请日：2020-06-24

申请人： INTERNATIONAL BUSINESS MACHINES CORPORATION

发明人： Chai Wah Wu ,  Oguzhan Murat Onen ,  Tayfun Gokmen ,  Malte Johannes Rasch ,  Mark S. Squillante ,  Tomasz J. Nowicki ,  Wilfried Haensch ,  Lior Horesh ,  Vasileios Kalantzis

IPC分类号： G06F17/16 ,  G06F7/78

摘要： A computer-implemented method for Eigenpair computation is provided. The method includes computing, her a hardware processor, an Eigenvector and respective Eigenvalues of the Eigenvector of a matrix by using a modified Stochastic Optimization process including performing a matrix vector product on a Resistive Processing Unit (RPU) crossbar array operatively coupled to the hardware processor and performing a scalar vector product on a digital device operatively coupled to the hardware processor and representing, for each of an Eigenpair, an initial guess for the Eigenvector and the respective Eigenvalues. The computing step includes storing the matrix in the RPU crossbar array.