Methods, systems, and processor-readable media for video anomaly detection based upon a sparsity model. A video input can be received and two or more diverse descriptors of an event can be computed from the video input. The descriptors can be combined to form an event matrix. A sparse reconstruction of the event matrix can be performed with respect to an over complete dictionary of training events represented by the diverse descriptors. A step can then be performed to determine if the event is anomalous by computing an outlier rejection measure on the sparse reconstruction.