The invention provides a system for controlling an irradiating direction of a vehicle lamp, which can cause the irradiating direction of the vehicle lamp to reliably point to the approximately straight-ahead direction as being restored after a failure thereby enhancing safety. The system includes a target swivel angle calculating section (11) to calculate a target swivel angle based on a vehicle steering angle, swivel drivers (12, 20) to execute, based on the target swivel angle, a swivel control to swivel the irradiating direction of the lamps (LHL, RHL), and a fail-safe controller (13) to execute, upon detection of a failure detected by a failure detector (23) in the system, a fail-safe operation to stop the swivel control. When the system is restored after the failure, the fail-safe controller determines whether the target swivel angle is within a straight-ahead range that includes an angle corresponding to a straight-ahead direction of a vehicle. If the fail-safe controller determines that the target swivel angle is within the straight-ahead range, the fail-safe controller deactivates the fail-safe operation to restart the swivel control.