公开(公告)号：US20220222523A1

公开(公告)日：2022-07-14

申请号：US17206164

申请日：2021-03-19

Applicant: Korea Advanced Institute of Science and Technology

Inventor： Hoi Jun YOO ,  Dong Hyeon HAN

IPC: G06N3/08 ,  G06N3/04

Abstract: Disclosed herein are an apparatus and method for training a low-bit-precision deep neural network. The apparatus includes an input unit configured to receive training data to train the deep neural network, and a training unit configured to train the deep neural network using training data, wherein the training unit includes a training module configured to perform training using first precision, a representation form determination module configured to determine a representation form for internal data generated during an operation procedure for the training and determine a position of a decimal point of the internal data so that a permissible overflow bit in a dynamic fixed-point system varies randomly, and a layer-wise precision determination module configured to determine precision of each layer during an operation in each of a feed-forward stage and an error propagation stage and automatically change the precision of a corresponding layer based on the result of determination.

公开(公告)号：US20210056427A1

公开(公告)日：2021-02-25

申请号：US16988737

申请日：2020-08-10

Applicant: Korea Advanced Institute of Science and Technology

Inventor： Hoi Jun YOO ,  Dong Hyeon HAN

IPC: G06N3/08 ,  G06N5/04

Abstract: Disclosed herein are an apparatus and method for training a deep neural network. An apparatus for training a deep neural network including N layers, each having multiple neurons, includes an error propagation processing unit configured to, when an error occurs in an N-th layer in response to initiation of training of the deep neural network, determine an error propagation value for an arbitrary layer based on the error occurring in the N-th layer and directly propagate the error propagation value to the arbitrary layer, a weight gradient update processing unit configured to update a forward weight for the arbitrary layer based on a feed-forward value input to the arbitrary layer and the error propagation value in response to the error propagation value, and a feed-forward processing unit configured to, when update of the forward weight is completed, perform a feed-forward operation in the arbitrary layer using the forward weight.