The invention discloses a small-sized abrasive belt type steel rail derusting cleaning device. The device is characterized by comprising a main bracket, travel wheels, a contact wheel and a micro-feed mechanism for adjusting the contact wheel in an abrasive belt running mechanism, wherein a circumferential groove is formed in the circumferential surface of each travel wheel; the width of the groove in the back travel wheel is greater than that of the groove in the front travel wheel; the micro-feed mechanism comprises a traveling nut, a fixed nut bracket and a screw rod; the abrasive belt running mechanism comprises a driving wheel, a driven wheel and the contact wheel which are driven by a motor; the driven wheel is arranged on a moving frame; the moving frame is driven to be lifted up and down by a horizontal push rod and a spring and can be positioned; and an abrasive belt is wound around the driving wheel, the driven wheel and the contact wheel. The device is convenient to use, high in polishing precision and efficiency and high in reliability.