A hybrid transmission/auto-conversion 3D format and scheme for transmission of 3D data towards various types of 3D displays is described. In the decoder (20) a stereo-to-depth convertor (24) generates a depth map. In the 3D video signal additional depth information called depth helper data (DH-bitstr) is sparsely transmitted both in time (partial depths in time) and/or spatially (partial depth within the frames). A depth switcher (25) selects the partial depths based on an explicit or implicit mechanism for indicating when these are to be used or when the depths must be automatically generated locally. Advantageously disturbing depth errors due to said stereo-to-depth convertor are reduced by the depth helper data.