Disclosed is an apparatus and method to determine usage rules for video content by buffer tracking. A computing device may include a secure processor configured to: store digital rights management (DRM) rules associated with a DRM key and usage rules for a session; command a cryptoprocessor to decrypt video content with the DRM key and to log an output buffer designation of the command to decrypt the video content. The secure processor may command a buffer tracking table to store the output buffer designation of the cryptoprocessor of the command to decrypt and the associated usage rules and a plurality of input and output buffer designations from a plurality of video content drivers, such that, based upon a buffer designation from an output driver to display video content received by the secure processor, the secure processor may determine usage rules to be applied to the decrypted video content for display.