A non-volatile storage apparatus comprises a plurality of memory cells that store host data and two models, a control circuit for writing to and reading from the memory cells, and an inference circuit. The inference circuit uses the first model with a first set of one or more metrics describing current operation of the non-volatile storage apparatus to make a first level prediction about defects and uses the second model with a second set of one or more metrics describing current operation of the non-volatile storage apparatus to make a second level prediction about defects. In one embodiment, the first level prediction is faster to make and uses less data collection, but is not as reliable, as the second level prediction. While second level prediction is more reliable, it takes more time to perform and requires a more intensive data collection, so it is only used when needed.