Compositions and methods are provided for delayed breaking of viscoelastic surfactant gels inside subterranean formations. Breaking is accomplished without mechanical intervention or use of a second fluid. The delayed breaking agent is a hydrophobically modified alkali swellable emulsion polymer, which can be a copolymer comprising acidic monomers, nonionic monomers, and associative monomers. The viscoelastic surfactant can be a zwitterionic surfactant, and can be selected from the group consisting of sultaines, betaines, and amidoamine oxides.