A multi-model medical scan analysis system is operable to generate a generic model by performing a training step on image data of a plurality of medical scans and corresponding labeling data. A plurality of fine-tuned models corresponding to one of a plurality of abnormality types can be generated by performing a fine-tuning step on the generic model. Abnormality detection data can be generated for a new medical scan by performing utilizing the generic model. One of the plurality of abnormality types is determined to be detected in the new medical scan based on the abnormality detection data, and a fine-tuned model that corresponds to the abnormality type is selected. Additional abnormality data is generated for the new medical scan by utilizing the selected fine-tuned model. The additional abnormality data can be transmitted to a client device for display via a display device.