With respect to a charged particle beam device, the step size of focal point measure for executing autofocusing is optimized to a value that is optimal with respect to the spread of an approximation curve for a focal point measure distribution. The step size of focal point measure for executing autofocusing is corrected using an image feature obtained based on a layout image derived from an image obtained at a first magnification or from design data. Autofocusing is executed based on the obtained step size to carry out observation, measurement, or to image the sample under inspection.