Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for testing the compatibility of client devices with web services. In one aspect, a method includes receiving a request from a first client device; obtaining a response to the request from a web service, wherein the web service provides responses according to a web service schema that defines a plurality of possible response formats, and wherein the response conforms to a first response format of the plurality of possible response formats; selecting a second, different response format of the plurality of possible response formats defined by the web service schema; modifying the response to generate a modified response, wherein the modified response conforms to the second, different response format; and sending the modified response to the first client device.