The voltage present at an input is chopped by a switch which is actuated by a pulse-width modulator controlled by the output voltage and is smoothed at least by a low-pass filter, consisting of an inductance and a capacitance, at the output. In order to reduce to a minimum the delay between the time an input voltage is applied and the time a rated value is at least approximately reached by the output voltage, the inductance is bypassed by a bypassing circuit which has a low inductance and which conducts if the output voltage is below a minimum voltage and in other cases is turned off. It contains, for example, a switch which is constructed as a switchable current source and which is actuated by a comparator comparing the output voltage with a reference voltage. In transformer-coupled embodiments, isolation between the output and the input is possible. The supply unit is suited, for example for supplying auxiliary devices of an instrument transformer from the current of the power-line to be monitored.