A system and methods are provided for decision making under uncertainty, for selecting an optimal action from among multiple candidate actions in belief space planning (BSP), including, for a new propagated belief: selecting from stored propagated beliefs a closest propagated belief; selecting, from among the stored measurement samples associated with the closest propagated belief, re-use measurement samples that provide a representative set of a measurement likelihood distribution corresponding to the new propagated belief; determining an information gap between a stored posterior belief associated with a re-use measurement sample and a new posterior belief that would be inferred by applying the re-use measurement sample to the new propagated belief; updating the stored posterior belief to account for the information gap; and calculating objective values for the multiple candidate actions, the objective values being weighted summations including immediate scores of the updated posterior beliefs.