A vehicle control system controls the speed of a vehicle according to actual road conditions and state of the vehicle. The control system acquires a present value of lateral acceleration occurring in a direction lateral to a controlled vehicle when the controlled vehicle avoids a collision with an obstacle or passes the obstacle. A target relative speed between the obstacle and the controlled vehicle is calculated from the present value of lateral acceleration and a turning radius of the controlled vehicle required for the controlled vehicle to avoid a collision with the obstacle or to pass the obstacle. Then, deceleration control is executed based on target relative deceleration rate at which the controlled vehicle is to be decelerated to reach the target relative speed.