A processor employs a hardware encryption module in the memory access path between an input/out device and memory to cryptographically isolate secure information. In some embodiments, the encryption module is located at a memory controller of the processor, and each memory access request provided to the memory controller includes VM tag value identifying the source of the memory access request. The VM tag is determined based on a requestor ID identifying the source of the memory access request. The encryption module performs encryption (for write accesses) or decryption (for read accesses) of the data associated with the memory access based on an encryption key associated with the VM tag.