A method for medical image segmentation includes accessing and updating a knowledge-base. First, a medical image is received and a sparse landmark signature is computed based on the medical image. Next, either a representative or a cohort average reference image set is selected. A portion of either representative reference image set or the cohort average reference image set is deformed to generate mappings to the medical image set. A segmentation for each structure of interest of the medical image set is determined. The knowledge-base is searched for representative matches to form a plurality of sub-volume base sets comprising a plurality of reference image set sub-volumes. A portion of the plurality of reference image set sub-volumes of the plurality of sub-volume base sets is deformed to generate mappings from the plurality of sub-volume base sets to corresponding structures of interest of the medical image set. A weighted-average segmentation for the plurality of structures of interest in the medical image set is calculated.