A system and method for real world autonomous vehicle trajectory simulation may include: receiving training data from a data collection system; obtaining ground truth data corresponding to the training data; performing a training phase to train a plurality of trajectory prediction models; and performing a simulation or operational phase to generate a vicinal scenario for each simulated vehicle in an iteration of a simulation. Vicinal scenarios may correspond to different locations, traffic patterns, or environmental conditions being simulated. Vehicle intention data corresponding to a data representation of various types of simulated vehicle or driver intentions.