A user identification system is provided, which includes an in-vehicle apparatus used by multiple users and a vehicular portable device carried by a user acting as a driver and associated with the in-vehicle apparatus. The vehicular portable device detects received signal strength of signals received from the portable terminals at multiple time points, and based on the received signal strength detected by the received signal strength detector, determines a driver terminal which is the portable terminal carried by the user acting as the driver. The vehicular portable device determines as the driver terminal the portable terminal that provides a smallest degree of change of the received signal strength detected multiple times.