A stereo image processing apparatus 1 specifies a line pair having a high correlation between one image and the other image of a stereo image, for an image region which has a predetermined width and is parallel to a scanning direction (X direction) in a stereo image which shoots a predetermined region from two different positions and has a scanning line direction and an epipolar line direction matched with each other, obtains a shift (vertical azimuth difference) in a vertical direction (Y direction) between the one image and the other image of the stereo image according to each position of the specified line pair, and corrects the stereo image so as to eliminate the obtained shift.