A method of manufacturing a semiconductor device according to an embodiment includes: forming a plurality of semiconductor layers located at a distance from one another on a first insulating film; forming a gate insulating film that covers both side faces and an upper face of each of the semiconductor layers; forming a gate electrode of a polysilicon film to cover the gate insulating film of each of the semiconductor layers; forming a second insulating film on an entire surface; exposing an upper face of the gate electrode by performing selective etching on a portion of the second insulating film; siliciding the gate electrode; and forming a stress applying film that applies a stress in a direction perpendicular to the extending direction of each of the semiconductor layers and parallel to an upper face of the first insulating film.