Structures and formation methods of a semiconductor device structure are provided. The semiconductor device structure includes a fin structure over a semiconductor substrate. The semiconductor device structure also includes a gate stack covering a portion of the fin structure. The semiconductor device structure further includes a spacer element over a sidewall of the gate stack. The spacer element includes a first layer and a second layer over the first layer. The dielectric constant of the first layer is greater than the dielectric constant of the second layer.