Aspects of the subject technology relate to systems, methods, and computer-readable media for performing ground segmentation through formation of a single super voxel. Image data tracking a surface as an object moves relative to the surface is accessed. Samples of the image data is grouped into different subsets to form a plurality of subsets of the image data. A normal vector to a corresponding plane defined by each subset of the plurality of subsets of the image data are identified. The samples of the image data are re-grouped into a single subset of the image data based on the corresponding normal vector for each subset of the plurality of subsets of the image data. A surface plane representation of the surface is identified based on the single subset of the image data.