The quality and robustness of audio echo cancellation is enhanced by selectively applying glitch recovery processes based on a quality measurement of the relative offset between capture and render audio streams. For example, large and small glitch detection is enabled for low relative offset variance; large glitch detection is enabled in a medium range of relative offset variance; and neither enabled at high variance. Further, a fast glitch recovery process suspends updating the adaptive filter coefficients of the audio echo cancellation while buffers are re-aligned to recover from the glitch, so as to avoid resetting the adaptive filter. When clock drift exists between capture and render audio streams, a multi-step compensation method is applied to improve AEC output quality in case the drifting rate is low; and a resampler is used to compensate the drift in case the drifting rate is high. An anti-clipping process detects clipping of the signals, and also suspends adaptive filter updating during clipping.