A phase difference AF processing unit (19) compares subject images between a region R1 and a region Rj (j=2 to m) in an AF area (53), and determines a phase difference detection pixel (52A, 52B) as a detection signal addition target with respect to a phase difference detection pixel (52A, 52B) in the region R1 among phase difference detection pixels (52A, 52B) in the region Rj. Further, the phase difference AF processing unit (19) adds up detection signals with respect to the phase difference detection pixels (52A, 52B) in the region R1 and the phase difference detection pixels (52A, 52B) which are addition targets, and generates a defocus amount (Df1) from a result of a correlation operation using detection signals after addition. A system control unit (11) drives a focus lens according to the defocus amount (Df1) to perform a focusing control.