A method, computer program product, and computer system, for receiving a first set of ground truth instances from a first source. A second set of ground truth instances may be received from a second source. The first and second sets of ground truth instances may be weighed differently based on a level of trust associated with each of the first and second sources. The weighted first and second sets of ground truth instances may be applied in a machine learning task executed by a computer.