The invention discloses a quaternion based object optical flow tracking method in the technical field of video image processing. In the method, color is processed and an optical flow is estimated in an entire signal mode by utilizing quaternion expression of color. Through the method, more accurate optical flow estimation can be achieved at a pixel position with space color change, so that characteristic points on an object can be more robustly tracked to reduce tracking errors. In the method, quaternion color corners are also used as reliable characteristic points during tracking. A good characteristic point set is formed by using the quaternion color corners and gray value corners together; and the object optical flow tracking is performed by combining a quaternion optical flow estimation algorithm.