A pipeline in a processor core includes: at least one stage that decodes instructions including load instructions that retrieve data stored at respective virtual addresses, at least one stage that issues at least some decoded load instructions out-of-order, and at least one stage that initiates at least one prefetch operation. Copies of page table entries mapping virtual addresses to physical addresses are stored in a TLB. Managing misses in the TLB includes: handling a load instruction issued out-of-order using a hardware page table walker, after a miss in the TLB, handling a prefetch operation using the hardware page table walker, after a miss in the TLB, and handling any software-calling faults triggered by out-of-order load instructions handled by the hardware page table walker differently from any software-calling faults triggered by prefetch operations handled by the hardware page table walker.