The present application provides a passive holding device for handling a trocar and a modular surgical system. The passive holding device has a fastening device with a coupling portion. A pivoting mechanism is connected to the coupling portion via a rotary joint. The pivoting mechanism has two L-shaped branches arranged parallel to one another, each having a short leg that is connected to a long leg via a pivot joint. The short leg of the first branch is longer than the short leg of the second branch and is connected to the long leg via a rotary shaft. The trocar holder has a gripping device and has an instrument brake. A tilting brake is arranged at the coupling portion, operatively connected to the rotary joint, and a pivoting brake is arranged at the coupling portion operatively connected to at least one of the terminal pivot joints.