公开(公告)号：US10817783B1

公开(公告)日：2020-10-27

申请号：US16868936

申请日：2020-05-07

Applicant: Facebook, Inc.

Inventor： Nadav Rotem ,  Abdulkadir Utku Diril ,  Mikhail Smelyanskiy ,  Jong Soo Park ,  Christopher Dewan

IPC: H04L29/06 ,  G06N3/08 ,  G06N3/10

Abstract: The disclosed computer-implemented method for efficiently updating neural networks may include (i) identifying a neural network that comprises sets of interconnected nodes represented at least in part by a plurality of matrices and that is trained on a training computing device and executes on at least one endpoint device, (ii) constraining a training session for the neural network to reduce the size in memory of the difference between the previous values of the matrices prior to the training session and the new values of the matrices after the training session, (iii) creating a delta update for the neural network that describes the difference between the previous values and the new values, and (iv) updating the neural network on the endpoint device to the new state by sending the delta update from the training computing device to the endpoint computing device. Various other methods, systems, and computer-readable media are also disclosed.

公开(公告)号：US10699190B1

公开(公告)日：2020-06-30

申请号：US15911120

申请日：2018-03-04

Applicant: Facebook, Inc.

Inventor： Nadav Rotem ,  Abdulkadir Utku Diril ,  Mikhail Smelyanskiy ,  Jong Soo Park ,  Christopher Dewan

IPC: G06F21/60 ,  H04L29/06 ,  G06N3/08 ,  G06N3/10

Abstract: The disclosed computer-implemented method for efficiently updating neural networks may include (i) identifying a neural network that comprises sets of interconnected nodes represented at least in part by a plurality of matrices and that is trained on a training computing device and executes on at least one endpoint device, (ii) constraining a training session for the neural network to reduce the size in memory of the difference between the previous values of the matrices prior to the training session and the new values of the matrices after the training session, (iii) creating a delta update for the neural network that describes the difference between the previous values and the new values, and (iv) updating the neural network on the endpoint device to the new state by sending the delta update from the training computing device to the endpoint computing device. Various other methods, systems, and computer-readable media are also disclosed.