A method is provided for fabricating a semiconductor device that includes providing a semiconductor substrate having a first region and a second region, forming a high-k dielectric layer over the substrate, forming a silicon layer over the high-k dielectric layer, forming a hard mask layer over the silicon layer, patterning the hard mask layer, silicon layer, and high-k dielectric layer to form first and second gate structures over the first and second regions, respectively, forming a contact etch stop layer (CESL) over the first and second gate structures, modifying a profile of the CESL by an etching process, forming an inter-layer dielectric (ILD) over the modified CESL, performing a chemical mechanical polishing (CMP) on the ILD to expose the silicon layer of the first and second gate structures, respectively, and removing the silicon layer from the first and second gate structures, respectively, and replacing it with metal gate structures.