A method of manufacturing a semiconductor device, including: forming a surface structure including a metal oxide semiconductor structure in a semiconductor substrate; forming an interlayer insulating film covering the surface structure; forming, in the interlayer insulating film, a contact hole to expose the surface structure; forming a first Al alloy film in contact with the surface structure and covering an entire surface of the interlayer insulating film, the first Al alloy film having a thickness of 0.05 μm to 0.5 μm; detecting, using a dark field, a defect of a surface of the first Al alloy film; forming a second Al alloy film covering the entire surface of the first Al alloy film; patterning the first and second Al alloy films; annealing the first and second Al alloy films; and dicing the semiconductor substrate into a plurality of chips, and picking, among the plurality of chips, chips free of the detected defect.