A position and orientation measurement apparatus comprises: a distance information obtaining unit adapted to obtain distance information of a target object captured by a capturing unit; a grayscale image obtaining unit adapted to obtain a grayscale image of the target object; a first position and orientation estimation unit adapted to estimate a position and orientation of the target object based on the information of a three-dimensional shape model and the distance information; a second position and orientation estimation unit adapted to estimate a position and orientation of the target object based on a geometric feature of the grayscale image and projection information obtained by projecting, on the grayscale image, the information of the three-dimensional shape model; and a determination unit adapted to determine whether a parameter of the capturing unit is needed to be calibrated or not, based on both a first and second estimated values.