A system and method for analyzing and visualizing local clinical features includes access of a medical image dataset comprising image data acquired from a patient and identification of a region of interest (ROI) dataset corresponding to an ROI from the medical image dataset. The system also includes application of an automated algorithm to the ROI dataset, identification of an intermediate result used by the automated algorithm to analyze the ROI, and access of reference data corresponding to the intermediate result, the reference data derived from a reference dataset and representing an expected behavior of the intermediate result. Further, the system includes comparison of the intermediate result to the reference data, generation of a deviation metric based on the comparison, the deviation metric representing a deviation of the intermediate result, and creation of a visual representation of the deviation metric.