A method for estimating camera orientation relative to a ground surface. Line segments are detected from an image captured by a camera. A first virtual cube having three orthogonal vanishing points with a random 3D orientation is superimposed on to the image. The line segments of the image are classified grouped into 3D-directional groups. A second virtual cube is superimposed on to the image with an initial 3D orientation. An optimal 3D orientation of the second virtual cube is computed by iteratively changing the 3D orientation of the second virtual cube and measuring perpendicular distances of the three orthogonal vanishing points to the three line segment groups in each iteration starting with the initial 3D orientation, wherein the optimal 3D orientation of the second virtual cube being one that provides shortest perpendicular distances. Co-variances of the orthogonal vanishing points of the second virtual cube at the optimal orientation are computed. Ground orientation is computed from the second virtual cube at the optimal orientation.