Memory objects associated with a portion of a cache (e.g., data blocks of a media file) are assigned a value based on their importance to an application that is consuming memory objects. The values are used to assign the data blocks to purge groups. The purge groups are a labeling mechanism for determining a purge order. A memory object associated with a first data block assigned to a first purge group may be purged before a memory object associated with a second data block assigned to a second purge group. As new data blocks are received by the application (e.g., from disk or a network connection), the blocks are assigned a value and added to a purge group. In some cases, the data blocks arrive out of order (e.g., order of consumption). Memory objects can be reassigned to a different purge group when new data blocks are added or reclaimed.