A technique for semantic video compression is shown in block (120). Uncompressed video data (210), including a plurality of video data segments (S1, S2, . . . Sn), are organized into two or more buffer slots (220), such that each of the two or more buffer slots is filled with one or more of the received video data segments, thereby forming two or more buffered video portions corresponding to the two or more buffer slots. The buffered video data is then processed by a leaking rule, to extract one or more buffered video portions, while outputting one or more non-extracted buffered video portions, as compressed video data (230). The leaking rule data is stored in a histogram (240) and later used to organize and index data according to a users request.