An image processing apparatus includes an image processing unit configured to generate a plurality of processed images by performing image processing on a plurality of input images, a determination unit configured to determine, as a reference image for encoding a processed image generated by the image processing on an input image, whether the input image corresponding to the processed image is to be used or another processed image is to be used, based on a parameter relating to the image processing, and an encoding unit configured to encode the processed image by referring to a reference image according to a determination by the determination unit.