A test controller submits testing phrases to a text classifier and receives, from the text classifier, classification labels each comprising one or more respective heatmap values each associated with a separate word. The test controller aligns each of the classification labels corresponding with a respective testing phrase. The test controller identifies one or more anomalies of a selection of one or more classification labels that are different from an expected classification label for the respective testing phrase. The test controller outputs a graphical representation in a user interface of the selection of one or more classification labels and one or more respective testing phrases with visual indicators based on one or more respective heatmap values.