A method for determining a semantic concept classification for a digital video clip, comprising: receiving an audio-visual dictionary including a plurality of audio-visual grouplets, the audio-visual grouplets including visual background and foreground codewords, audio background and foreground codewords, wherein the codewords in a particular audio-visual grouplet were determined to be correlated with each other; analyzing the digital video clip to determine a set of visual features and a set of audio features; determining similarity scores between the digital video clip and each of the audio-visual grouplets by comparing the set of visual features to any visual background and foreground codewords associated with a particular audio-visual grouplet, and comparing the set of audio features to any audio background and foreground codewords associated with the particular audio-visual grouplet; and determining one or more semantic concept classifications using trained semantic classifiers.