A system and method for detecting a seat belt violation in a vehicle. The method may include receiving an image including a region of interest where a seat belt is visible if the seat belt is buckled. One or more first features may be extracted from the image. The one or more first features may be compared to one or more second features extracted from a plurality of sample images. It may be determined whether the seat belt is buckled in the image based upon the comparison.