Embodiments are directed towards multi-level entity classification. An object associated with an entity is received. In one embodiment the object comprises and email and the entity comprises the IP address of a sending email server. If the entity has already been classified, as indicated by an entity classification cache, then a corresponding action is taken on the object. However, if the entity has not been classified, the entity is submitted to a fast classifier for classification. A feature collector concurrently fetches available features, including fast features and full features. The fast classifier classifies the entity based on the fast features, storing the result in the entity classification cache. Subsequent objects associated with the entity are processed based on the cached result of the fast classifier. Then, a full classifier classifies the entity based on at least the full features, storing the result in the entity classification cache.