An image synthesis apparatus for generating a composite image by synthesizing a second image with a first image, includes an image acquisition unit configured to acquire first group images including the first image and a plurality of different resolution images corresponding to the first image, and second group images including the second image and a plurality of different resolution images corresponding to the second image, a frequency component calculation unit configured to calculate low-frequency components and high-frequency components from the first group images and the second group images, and an image synthesis unit configured to generate the composite image on the basis of the low-frequency components of the first group images and the high-frequency components of the second group images.