A bottle or similar container is provided with a safety closure comprising a push-on cap having a sealing plug on its base penetrating into the bottle mouth and having projections on the inside of its peripheral wall which engage behind an annular beading on the bottle neck to retain the cap on the bottle. In addition to the beading, mutually opposite cams project from the bottle neck each providing an upper surface which ascends in a circumferential direction towards the bottle mouth. The peripheral wall of the cap, which is made of a slightly flexible material, is spaced from the bottle neck and at mutually opposite positions on its inside there are inwardly projecting cam followers which clear the cams when the cap is rotated normally but which engage and ride up the upper cam surfaces when the cap is pressed inwards in the regions of the followers and the cap is simultaneously rotated in the same direction as the cam surfaces ascend, thereby pushing the cap off the bottle.