A moving image coder has a forward predicting motion vector/prediction error amount detecting section for detecting an optimum motion vector from a past frame, making a predicted image and calculating a prediction error amount from the present frame. The coder also has a backward predicting motion vector/prediction error amount detecting section for detecting an optimum motion vector from a future frame, making a predicted image and calculating a prediction error amount from the present frame. The coder further has a comparator/mode selecting section for inputting the prediction error amounts from the forward predicting motion vector/prediction error amount detecting section and the backward prediction motion vector/prediction error amount detecting section, making a comparative calculation of the input prediction error amounts, and selecting one of a forward prediction mode, a backward prediction mode and a bidirectional prediction mode.