Defect classification includes acquiring one or more images of a specimen, receiving a manual classification of one or more training defects based on one or more attributes of the one or more training defects, generating an ensemble learning classifier based on the received manual classification and the attributes of the one or more training defects, generating a confidence threshold for each defect type of the one or more training defects based on a received classification purity requirement, acquiring one or more images including one or more test defects, classifying the one or more test defects with the generated ensemble learning classifier, calculating a confidence level for each of the one or more test defects with the generated ensemble learning classifier and reporting one or more test defects having a confidence level below the generated confidence threshold via the user interface device for manual classification.