An augmented reality (AR) system receives image data from a camera of mobile device. The image data includes images of objects in the field of view of and at least two markers. A repository contains position information for objects and markers that may potentially appear in an image. The AR system receives a confidence level for the markers in the image, and selects the marker with the highest confidence level. The transformation and position of the marker in the image data is used to determine the identity of objects in the field of view. Further, the AR system generates overlaid image data that can be incorporated into the image data and sent back to be displayed on the mobile device. The overlaid image data is positioned according to the position data for the object it is associated with, and the transformation and position data for the selected marker.