The efficacy of security products and practices is quantified, based on monitored activities and conditions on multiple computers over time. A set of metrics is defined, specifying what criteria concerning computer security systems are to be quantified. Telemetry data concerning the defined metrics are collected from multiple computers, such as the customer base of a security product vendor. Security configuration information such as the deployments and settings of security systems on computing devices is monitored. This monitored information tracks what security products are deployed on which machines, and how these products are configured and used. Collected telemetry is correlated with monitored configuration information, enabling determination of what security product deployments and settings are in place when specific security incidents, operations and other types of actions occur. The determined correlations are amalgamated, amalgamated correlation information is analyzed, and the efficacy of specific security products and configurations is quantified.