The present invention relates to an IP based or network based audio packet switching system, and more specifically to an audio packet switching system for integrating each independent module into a single interface for consistent control and data flow. Independent modules, such as a RTP/RTCP, encoder/decoder and mixer, are interworked with reception and transmission buffers to make a system more flexible. Additionally, audio data paths inside the switch are controlled through an audio packet router. The multi protocol system employing the RTP provides a single RTP control interface for transmitting and receiving audio packets to completely manage the system totally and raises a degree of resource utilization of the system by making it easy to implement additional functions to the RTP and coder through the buffers.