A computer system uses Learning from Demonstration (LfD) techniques in which a multitude of tasks are demonstrated without requiring careful task set up, labeling, and engineering, and learns multiple modes of behavior from visual data, rather than averaging the multiple modes. As a result, the computer system may be used to control a robot or other system to exhibit the multiple modes of behavior in appropriate circumstances.