This apparatus has plural fixed grinding machines, which are installed along the direction of travelling line of billets. Facing pairs of grinders are equipped in a manner that the distances between each facing pair of grinders are gradually shorter along the travelling direction of the billets from the upstream side to the downstream side. In such a multiple-stage grinding machine, the burr on the welded part of the top and bottom side of the billet is ground sequentially by the grinders, and the burr on welded part of the right and left side of the billet is ground sequentially by the grinders.