A computerized method is disclosed for generating a prioritized listing of alerts based on scoring by a machine learning model and retraining the model based on user feedback. Operations of the method include receiving a plurality of alerts, generating a score for each of the plurality of alerts through evaluation of each of the plurality of alerts by a machine learning model, generating a prioritized listing of the plurality of alerts based on the generated scores, receiving user feedback on the prioritized listing, retraining the machine learning model based on the user feedback by generating a set of labeled alert pairs, wherein a labeled alert pair includes a first alert, a second alert, and an indication as to which of the first alert or the second alert is a higher priority in accordance with the user feedback, and evaluating subsequently received alerts with the retrained machine learning model.