In a network having an L2SW in which uplink VLAN is set, a transfer delay of a voice call packet is prevented. In a process of establishing a session between terminals ( 5 ), a packet transfer device ( 1 ) stores information capable of specifying a sending terminal ( 5 - 1 ) and a receiving terminal ( 5 - 2 ) and a correspondence relation (user connection information) to a connection port into a table. A SIP message of a session control packet communicated between the terminals ( 5 ) is snooped, and a destination IP address, a transmission source IP address, a destination port number, a transmission source port number, a connection port etc. are stored into the table. In a case where an RTP packet as voice data is received after the session is established, the packet transfer device ( 1 ) performs a loopback transfer or transfers it to an uplink port in accordance with the table.