A “texture generator” uses an inverse texture synthesis solution that runs in the opposite direction to traditional forward synthesis techniques to construct 2D texture compactions for use by a graphics processing unit (GPU) of a computer system. These small 2D texture compactions generally summarize an original globally variant texture or image, and are used to reconstruct the original texture or image, or to re-synthesize new textures or images under user-supplied constraints. In various embodiments, the texture generator uses the texture compaction to provide real-time synthesis of globally variant textures on a GPU, where texture memory is generally too small for large textures. Further, the texture generator provides an optimization framework for inverse texture synthesis which ensures that each input region is properly encoded in the output compaction. In addition, the texture generator also computes orientation fields for anisotropic textures containing both low- and high-frequency regions.