At the communicating time of a fuel discharge passage (C3) by a control valve (48), fuel flows from an outer control chamber (R2o) via a communication passage (47) into an inner control chamber (R2i), and flows from the inner control chamber (R2i) to a fuel tank (T). As a result of the fuel flow in the communication passage (47), a differential pressure (Pco > Pci) is produced between an inner control pressure (Pci) and an outer control pressure (Pco). As a result, the outer differential pressure just after the outer valve was opened can be set at a small value thereby to suppress the increase of the unburned HC quantity in the exhaust gases at a low-load time, as caused by the high rising rate of an outer needle valve (42) just after the outer valve was opened. Moreover, the inner differential pressure just after the inner valve was opened can be set at a large value thereby to suppress the increase of the smoke quantity in the exhaust gases, as caused by the low rising rate of an inner needle valve (43) just after the inner valve was opened.