A through silicon via structure is located in a recess of a substrate. The through silicon via structure includes a barrier layer, a buffer layer and a conductive layer. The barrier layer covers a surface of the recess. The buffer layer covers the barrier layer. The conductive layer is located on the buffer layer and fills the recess, wherein the contact surface between the conductive layer and the buffer layer is smoother than the contact surface between the buffer layer and the barrier layer. Moreover, a through silicon via process forming said through silicon via structure is also provided.