To avoid uniform deterioration of the voice quality of all the calls sharing an IP phone line at a time of congestion. The congestion control method in an IP phone system includes a call state acquisition process of acquiring the number of simultaneous call sessions in one line shared by a plurality of IP phones. A congestion control method further includes a packet loss control process of setting a priority ranking to each call session and dynamically controlling packet loss rates of the call sessions in accordance with the number of the simultaneous call sessions and the priority ranking. In the packet loss control process, the packet loss rate is set lower for call sessions ranked higher in priority, and higher for call sessions ranked lower in priority.