The conventional encoding method has a problem that if a backward reproduction is to be performed on a frame-by-frame basis, it requires a forward decoding to be performed in the first place, resulting in increases of processing amounts and memories. Pictures comprising only forward predictions and those comprising only backward predictions as to frames in an encoded stream are recorded, thereby providing streams that allow a backward reproduction to be easily performed on a frame-by-frame basis. Moreover, application of this to encoding of multi-viewpoint images realizes an apparatus that can change viewpoints in real time to reproduce images.