Rounding error can be reduced when evaluating binary floating point polynomials utilizing a Floating Point Unit (58) by first computing the sum of products of second and higher order polynomial terms. Next, the Floating Point Unit (58) adds a zero.sup.th level term to the product of a first order coefficient and an independent variable to form a "Big" term. The Floating Point Unit (58) calculates as a "Little" term the rounding error resulting from the computation of the "Big" term. The "Little" term is then added to the sum of products of higher order terms to form an "Intermediate" term. Finally, the Floating Point Unit (58) adds the "Big" term to the "Intermediate" term to form the polynomial result corrected by the rounding error introduced by the computation of the low order terms.