This invention discloses a fast and correct video coding method and a device thereof. After a coding history of some coding video frame units during a video process is checked, the sequence of each coding video frame unit of the coding frame, a frame-to-frame prediction, and an in-frame prediction is self-adaptive exchanged. In addtion, the calculation of the coding mode in the rear section of the calculation sequence be selectively skipped so as to quicken the coding process without reducing the video quality.