A semiconductor device includes a substrate, a first film stack, a second film stack, a first gate spacer, a buffer layer, and a second gate spacer. The first and second film stacks are located on the substrate, and are respectively located in an array area and a periphery area. The first gate spacer includes a first portion on a sidewall of the first film stack and a second portion on a sidewall of the second film stack. The buffer layer includes a first portion on a sidewall of the first portion of the first gate spacer and a second portion on a sidewall of the second portion of the first gate spacer. The second gate spacer includes a first portion on a sidewall of the first portion of the buffer layer and a second portion on a sidewall the second portion of the buffer layer.