The present invention discloses a method and an apparatus for recognizing video captions, wherein the method comprises: calculating the difference between two frame images having adjacent time of the same caption in a video to obtain a residual image; inverting the residual image to obtain an anti-residual image; performing a binarization processing on the anti- residual image; performing an and operation between the binarization processed anti-residual image and a current frame image of the same caption to obtain a caption image of the same caption; and performing character recognition on the caption image. The present invention can implement the processing of video contents to obtain the caption image, enable the caption image to achieve the degree of character recognition, and substantially increase the accuracy of video captions recognition results with regard to the existing method for recognizing video captions; and the present invention can also be applied to the sign language dubbing video processing of a digital television program video to implement real-time sign language translation processing for the digital television program contents in the case of the picture captions.