Systems and methods are provided for constructing, using, and updating the sparse map for autonomous vehicle navigation. A system may comprise a processor and a memory. The memory may include instructions, which when executed on the processor, cause the processor to maintain a map; determine, based on analysis of image data, an existence of a non-transient condition that is inconsistent with the map, the image data from a camera integrated with the autonomous vehicle; and update the map.