Examples described herein relate to transitioning a playback session between portable playback devices such as “smart” headphones, earbuds, and handheld speakers with playback devices of a zone-based media playback system. Exemplary techniques facilitate continuity of playback when transitioning between locations (e.g., from at home to on-the-go or vice versa) or between listening paradigms (e.g., personal or out-loud playback of audio content). An example implementation includes detecting a swap trigger, determining the source playback device(s) and target playback device(s), and performing a playback session swap between the source playback device(s) and target playback device(s).