In observation of a sample having a structure in its depth direction, a charged particle beam apparatus that can form an SEM image reflecting a sample shape at a desired depth by a single image acquisition while avoiding enlargement of the apparatus is provided. The apparatus has: an irradiation optical system for irradiating and scanning a charged particle beam generated from a charged particle source on the sample; a detection optical system having a detector that detects charged particles generated from the sample by the irradiation of the charged particle beam, and converts them into an electric signal at a predetermined sampling period; and an image processing unit for forming an image based on the electric signal from the detector, in which the image processing unit detects a peak of wave height values for each pixel from the electric signal at each sampling time, and forms the image based on the peak of the detected wave height values.