Machines, systems and methods for maintaining a representative data set in a document classification system, the method comprising: including an initial set of seed representative data in a representative data set (RDS) implemented for a knowledge base (KB), wherein the KB is trained to classify documents provided to a document classification system based on analysis of the representative documents included in the RDS and a set of rules, wherein the seed representative data includes a balanced number of representative data across a plurality of classes; updating the RDS by adding or removing representative data from the RDS based on feedback received about accuracy of classification of one or more documents by the classification system; and retraining the KB, wherein the retraining is performed based on occurrence of one or more events.