A system and method for completing risk object identification that include receiving image data associated with a monocular image of a surrounding environment of an ego vehicle and analyzing the image data and completing semantic waypoint labeling of at least one region of the surrounding environment of the ego vehicle. The system and method also include completing counterfactual scenario augmentation with respect to the at least one region. The system and method further include determining at least one driver intention and at least one driver response associated with the at least one region.