A lithographic apparatus includes a phase adjuster to adjust a phase of an optical wave traversing an optical element of the phase adjuster during exposure of a pattern on a substrate. In an embodiment, the optical element is a heat controllable optical element in a projection system of the lithographic apparatus. In use, the pattern is illuminated with an illumination mode including an off-axis radiation beam. This beam is diffracted into zeroth-order and first-order diffracted beams oppositely and asymmetrically inclined with respect to an optical axis. An area is identified where the first-order diffracted beam traverses the optical element. An image characteristic of an image of the pattern is optimized by calculating a desired optical phase of the first-order diffracted beam in relation to the optical phase of the zeroth-order diffracted beam. The phase adjuster is controlled to apply the desired optical phase to the first order diffracted beam.