A vision system suitable for use for a vehicle includes a plurality of exterior viewing cameras disposed at the vehicle and having respective fields of view exterior of the vehicle. A display system is operable to display images for viewing by the driver of the vehicle. Responsive at least in part to determination that the gaze direction of the driver of the vehicle is towards a non-transparent portion of the vehicle, the display system displays an image derived from image data captured by at least one of the plurality of exterior viewing cameras to display a representation of the driver's view as it would be through the non-transparent portion of the vehicle. The gaze direction of the driver may be determined by a monitoring system that processes image data captured by an interior viewing camera having a field of view that encompasses the head of the driver.