A hard tempered aluminum alloy sheet useful for deep drawing is prepared by a process comprising the steps of hot rolling an ingot of aluminum alloy consisting essentially of 4.0 to 6.0 Mg wt. %, at least one component selected from the group consisting of 0.1 to 0.7 wt. % Mn, 0.01 to 0.05 wt. % Zr, 0.005 to 0.12 wt. % V and up to 0.05 wt. % Ti and the balance aluminum to a sheet not more than 2.5 mm thick, intermediate annealing the hot rolled alloy and final cold rolling the annealed alloy to effect a rolling reduction of less than 85%.In case the finishing temperature of the hot rolling is adjusted to a temperature of 300.degree. to 350.degree. C., the above intermediate annealing can be omitted.