A method for forming a semiconductor device structure is provided. The method includes forming a plurality of first semiconductor layers and a plurality of second semiconductor layers on a substrate, and the first semiconductor layers and the second semiconductor layers are alternately stacked. The method also includes forming a dummy gate structure over the first semiconductor layers and the second semiconductor layers. The method further includes removing a portion of the first semiconductor layers and second semiconductor layers to form a trench, and removing the second semiconductor layers to form a recess between two adjacent first semiconductor layers. The method includes forming a dummy dielectric layer in the recess, and removing a portion of the dummy dielectric layer to form a cavity. The method also includes forming an inner spacer layer in the cavity.