An optical imaging system with dynamic buffer management is described. Embodiments of the optical imaging system include an image sensor, a buffer, and a buffer mode controller. The image sensor includes a pixel array to read out a plurality of pixel lines for a frame of an image. The buffer is configured to store a plurality of buffer values corresponding to a plurality of pixel values for each of the pixel lines. The buffer mode controller is configured to dynamically switch between an accumulation mode and an averaging mode. The accumulation mode facilitates a computation of each buffer value according to a sum value of the corresponding pixel value and a corresponding previous buffer value. The averaging mode facilitates a computation of each buffer value according to an average value of a normalized value of the corresponding pixel value and the corresponding previous buffer value.