Mechanisms for managing data segments in a tiered storage system are provided. The mechanisms maintain at least one counter for each data segment in the tiered storage system. Each counter in the at least one counter counts a number of access operations to a corresponding data segment for a predetermined time interval. The mechanisms further perform one or more analytical operations based on one or more values of the at least one counter for each data segment to make residence determinations for each data segment. The mechanisms also adjust a storage location of one or more data segments in tiers of the tiered storage system to thereby move the one or more data segments to appropriate tiers of the tiered storage system based on results of the one or more analytical operations.