Embodiments provide a position service experimentation system to enable comparison of modeling and inference methods as well as characterization of input datasets for correspondence to output analytics. Crowd-sourced positioned observations are divided into a training dataset and a test dataset. A beacons model is generated based on the training dataset, while device position estimations are calculated for the test dataset based on the beacons model. The device position estimations are compared to the known position of the computing devices generating the positioned observations to produce accuracy values. The accuracy values are assigned to particular geographic areas based on the position of the observing computing device and aggregated to enable a systematic analysis of the accuracy values based on geographic area and/or positioned observations characteristics.