An apparatus, method, and program product for optimizing a multiprocessor computing system by sampling memory reference latencies and adjusting components of the system in response thereto. During execution of processes the computing system, memory reference sampling of memory locations from shared memory of the computing system referenced in the executing processes is performed. Each sampled memory reference collected from sampling is associated with a latency and a physical memory location in the shared memory. Each sampled memory reference is analyzed to identify segments of memory locations in the shared memory corresponding to a sub-optimal latency, and based on the analyzed sampled memory references, the physical location of the one or more identified segments, the processor on which one or more processes referencing the identified segments, and/or a status associated with the one or more identified segments is dynamically adjusted to thereby optimize memory access for the multiprocessor computing system.