The present invention provides a scheme for compressing the color components of image data, and in particular, data used in multi-sampled anti-aliasing applications. Adjacent pixels are grouped into rectangular tiles, with the sample colors stored in compressed formats accessible via an encoded pointer. In one embodiment, duplicate colors are stored once. Unlike prior compression schemes that rely on pixel to pixel correlation, the present invention takes advantages of the sample to sample correlation that exists within the pixels. A memory and graphics processor configuration incorporating the tile compression schemes is also provided. The configuration defines the tile sizes in main memory and cache memory. In one embodiment, graphics processor relies on a Tile Format Table (TFT) to process incoming tiles in compressed formats. The present invention reduces memory consumption and speeds up essential and oft-repeated operations in rendering. Thus it is valuable in the design and manufacture of graphic sub-systems.