A registration image is aligned or registered with a reference image by user interaction. The registration image is segmented into a hierarchy of elements defining a mesh having more than one level. At each level, individual elements may be selected by the user, translated, rotated and scaled with respect to the remainder of the registration image and to the reference image. Refinement of the user registration is facilitated by allowing the user to define further levels in the segmentation where each of the elements at the further levels may be separately manipulated in a similar manner. Interpolation may be performed between the moved or altered elements and between the elements and the remainder of the registration image to obtain a more satisfactorily registered image. The technique may be used with other interpolation and registration processes.