A method of manufacturing a semiconductor device having metal gates and the semiconductor device are disclosed. The method comprises providing a first sacrificial gate associated with a first conductive type transistor and a second sacrificial gate associated with a second conductive type transistor disposed over the substrate, wherein the first conductive type and the second conductive type are complementary; replacing the first sacrificial gate with a first metal gate structure; forming a patterned dielectric layer and/or a patterned photoresist layer to cover the first metal gate structure; and replacing the second sacrificial gate with a second metal gate structure. The method can improve gate height uniformity during twice metal gate chemical mechanical polish processes.