The invention provides an image transmission apparatus and a method of controlling the image transmission apparatus. The image transmission apparatus includes a destination setting unit configured to set a transmission destination of image data, a transmission unit configured to transmit the image data to the destination set by the destination setting unit, an authentication unit configured to perform processing for authentication required for the transmission unit to transmit the image data to the destination set by the destination setting unit, a storage unit configured to store a destination table in which authentication information required for the authentication unit to perform the processing for authentication is registered in such a manner that the authentication information is associated with the destination, a selection unit configured to select one of a first authentication mode, in which the authentication unit performs the processing for authentication using the authentication information stored in the storage unit when the transmission unit transmits the image data to the destination set by the destination setting unit, and a second authentication mode, in which the authentication unit performs the processing for authentication using authentication information entered by a user when the transmission unit transmits the image data to the destination set by the destination setting unit, and a limitation unit configured to limit a number of destinations to which the transmission unit transmits the image data when the selection unit selects the second authentication mode.