A permanent magnet material is prepared by machining an anisotropic sintered magnet body having the compositional formula: Rx(Fe1-yCoy)100-x-z-aBzMa wherein R is Sc, Y or a rare earth element, M is Al, Cu or the like, to a specific surface area of at least 6 mm−1, heat treating in a hydrogen gas-containing atmosphere at 600-1,100° C. for inducing disproportionation reaction on the R2Fe14B compound, and continuing heat treatment at a reduced hydrogen gas partial pressure and 600-1,100° C. for inducing recombination reaction to the R2Fe14B compound, thereby finely dividing the R2Fe14B compound phase to a crystal grain size ≦1 μm.