Techniques for encoding a two-dimensional (2D) video sequence of video frames along with three-dimensional (3D) conversion information comprising a set of parameters that can be applied to each of the video frames of the 2D sequence to generate 3D video data are described. The set of parameters can be applied to each of the original video frames of the 2D sequence to generate secondary views of each of the original video frames. The original video frames and the secondary views may collectively define a stereoscopic 3D video sequence.