A semiconductor device includes a semiconductor substrate and a gate structure. The semiconductor substrate includes a first semiconductor fin and a second semiconductor fin. The gate structure includes a work function metal structure crossing over the first semiconductor fin and the second semiconductor fin. The work function metal structure comprises a first portion over a portion of the first semiconductor fin, a second portion over a portion of the second semiconductor fin, and a third portion connecting the first portion to the second portion, wherein a thickness of the third portion is smaller than a thickness of the second portion and greater than a thickness of the first portion along an extension direction of the second semiconductor fin.