Systems and methods for audio processing include capturing first sound data via at least one microphone of a network microphone device (NMD) and determining, via a voice activity detection process, that the first sound data does not include voice activity. The first sound data is stored in a buffer, and the NMD forgoes spatial processing of the first sound data. The NMD can capture second sound data and determine, via the voice activity process, that the second sound data includes voice activity. The NMD spatially processes the second sound data to produce filtered sound data. The NMD detects a wake word based on data in the buffer. After detecting the wake word, the NMD may determine an action to be performed based on the data in the buffer.