A defect analyzing method includes acquiring a position and a size of a defect obtained in a defect inspection of a semiconductor device and a waveform of a reflected light in a region which includes the defect, the waveform being obtained in an optical inspection; acquiring process step information which includes a plurality of process steps to manufacture the semiconductor device and a processing content per the process step; performing a process simulation of the semiconductor device based on the position and the size of the defect and the process step information; performing an optical simulation on a result of the process simulation thereby to generate a waveform of a reflected light; calculating a similarity degree between the acquired waveform of the reflected light and the generated waveform of the reflected light; and judging whether or not the calculated similarity degree exceeds a threshold value registered in advance.