Transactional memory accesses are tracked using read and write sets based on actual program flow. A read and write set is associated with a range of instructions of a transaction. When execution follows a predicted branch, loads and stores are marked as being of selected read and write sets. Then, when a misprediction is processed, and execution is rewound, speculatively added read and write set indications are removed from the read and write sets.