One embodiment relates to an apparatus using electrons for inspection or metrology of a semiconductor substrate. The apparatus includes an electron source, electron lenses, scan deflectors, an objective electron lens, a collection electron lens, a pin-hole filter, de-scan deflectors, and a detector. The collection electron lens is configured to focus the secondary electrons so as to form a secondary electron beam which is focused at a conjugate focal plane, and the pin-hole filter is positioned at the conjugate focal plane. The de-scan deflectors are configured to controllably deflect the secondary electrons so as to counteract an influence of the scan deflectors such that a center portion of the secondary electron beam passes through the filter and a remainder portion of the secondary electron beam is filtered out by the filter. Other embodiments and features are also disclosed.