A near-eye display system includes a display panel to display a lightfield image comprising an array of elemental images and a lenslet array facing the display panel, the lenslet array comprising lenslets with spatially-varying prescriptions, such as different diameters, different focal lengths, and different asphere terms. The lenslet array further may be formed with a planar or non-planar substrate. The system further may include a distortion map comprising a set of transform matrices, each transform matrix configured to pre-distort a corresponding elemental image of a lightfield image so as to compensate for distortion expected to be introduced by a corresponding lenslet of the lenslet array, as well as a rendering component configured to modify the array of elemental images of a rendered lightfield using the distortion map to generate a modified lightfield image, and provide the modified lightfield image for display at the display panel for viewing via the lenslet array.