A semiconductor device and a manufacturing method thereof are provided. The semiconductor device includes a substrate, a patterned conductive layer and an epitaxial layer. The substrate includes a first fin structure and a second fin structure respectively protruding from a top surface of the substrate, and the second fin structure has a recess. The patterned conductive layer is disposed on the substrate and covers a first end of the first fin structure. The epitaxial layer is disposed in the recess. The first end of the first fin structure and a second end of the epitaxial layer face a first direction.