Systems and methods are provided to perform weight update operations in a resistive processing unit (RPU) system to update weight values of RPU devices comprising tunable resistive device. A weight update operation for a given RPU device includes maintaining a weight update accumulation value for the RPU device, adjusting the weight update accumulation value by one unit update value in response to a detected coincidence of stochastic bits streams of input vectors applied on an update row and update column control lines connected to the RPU device, generating a weight update control signal in response to the accumulated weight value reaching a predefined threshold value, and adjusting a conductance level of the tunable resistive device by one unit conductance value in response to the weight update control signal, wherein the one unit conductance value corresponds to one unit weight value of the RPU device.