A decoder includes circuitry configured to receive a bitstream, decode a plurality of video frames from the bitstream, determine for a current block of a current frame that a long term reference block update mode is enabled, determine a long term reference block update including pixel values and using the plurality of video frames, and update a portion of a long term reference frame with the long term reference block update. Related apparatus, systems, techniques and articles are also described.