High-precision normal information on the surface of a subject is generated by capturing an image of the subject. A normal information generating device captures the image of the subject and thereby passively generates normal information on the surface of the subject. The normal information generating device includes: a stereo polarization image capturing section for receiving a plurality of polarized light beams of different polarization directions at different viewpoint positions and obtaining a plurality of polarization images of different viewpoint positions; and a normal information generating section for estimating a normal direction vector of the subject based on the plurality of polarization images of different viewpoint positions.