To provide an ion milling system that can suppress an orbital shift of an observation electron beam emitted from an electron microscope column, the ion milling system includes: a Penning discharge type ion gun 100 that includes a permanent magnet 114 and that generates ions for processing a sample; and a scanning electron microscope for observing the sample, in which a magnetic shield 172 for reducing a leakage magnetic field from the permanent magnet 114 to the electron microscope column is provided.