Systems and methods for a head tracked stereoscopic display system that uses light field type data may include receiving light field type data corresponding to a scene. The stereoscopic display system may track a user's head. Using the received light field type data and the head tracking, the system may generate three dimensional (3D) virtual content that corresponds to a virtual representation of the scene. The stereoscopic display system may then present the 3D virtual content to a user. The stereoscopic display system may present a left eye perspective image and a right eye perspective image of the scene to the user based on the position and orientation of the user's head. The images presented to the user may be updated based on a change in the position or the orientation of the user's head or based on receiving user input.