A method of forming a semiconductor device includes following steps. Firstly, a gate structure is formed on a substrate, and two source/drain regions are formed. Then, a contact etching stop layer (CESL) is formed to cover the source/drain regions, and a first interlayer dielectric (ILD) layer is formed on the CESL. Next, a replace metal gate process is performed to form a metal gate and a capping layer on the metal gate, and a second ILD layer is formed on the first ILD layer. Following these, a first opening is formed in the second and first ILD layers to partially expose the CESL, and a second opening is formed in the second ILD to expose the capping layer. Finally, the CESL and the capping layer are simultaneously removed.