An image processing device includes a distance data analysis unit that analyzes subject distance information with the partial region units of captured images having different viewpoints, and a data generation unit that determines whether a recorded image or an output image is set to a two-dimensional image or set to a three-dimensional image depending on the analysis result, and generates the recorded or output image based on the determination result, wherein the data generation unit determines a recall level of stereoscopic vision in three-dimensional display based on the analysis result, and if it is determined that the recall level of stereoscopic vision is low, generates a two-dimensional image as the recorded or output image, and if it is determined that the recall level of stereoscopic vision is high, generates a three-dimensional image as the recorded or output image.