The present disclosure provides a semiconductor device and a method of manufacturing the same. The semiconductor device includes a semiconductor substrate, a restraint layer, a plurality of contact plugs, and a plurality of through silicon vias. The restraint layer is disposed on the semiconductor substrate, and the contact plugs are inserted into the restraint layer. The through silicon vias extend from a bottom surface of the semiconductor substrate to a front surface opposite to the back surface and the through silicon vias are in contact with the contact plugs, respectively.