An image processing device includes a viewpoint separating unit configured to separate multi-viewpoint image data, including images of multiple viewpoints and representing intensity distribution of light and the direction of travel of light according to positions and pixel values of pixels, into a plurality of single-viewpoint image data for each of the individual viewpoints; and a parallax control unit configured to control amount of parallax between the plurality of single-viewpoint image data obtained by separation into individual viewpoints by the viewpoint separating unit.