What is disclosed is a system and method for compensating for motion during processing of a video of a subject being monitored for physiological function assessment. In one embodiment, image frames are received. Successive batches of N video frames are processed to isolate pixels associated with a body region of the subject where a physiological signal is registered by the camera. The pixels are processed to obtain a time-series signal for each batch. A determination is made whether movement during video acquisition of this batch of image frames exceeds a threshold level. If so then a size N of the next batch of image frames is changed to: N=N+M1, where N+M1≦Nmax. Otherwise, a size N of a next batch is changed to: N=N−M2, where N−M2≧Nmin. Thereafter, processing repeats in a real-time continuous manner as the next batch of the N image frames is received. Various embodiments are disclosed.