Google App Engine disregards Accept-Encoding
$ curl -H 'Accept-Encoding: gzip' -A 'Random/5.0' http://url |file - /dev/stdin: ASCII text, with very long lines, with no line terminators $ curl -H 'Accept-Encoding: gzip' -A 'Random/5.0 gzip' http://url |file - /dev/stdin: gzip compressed data, max compression
Google, in their infinite wisdom, have decided that the
Accept-Encoding header is essentially useless.
Why I don't know, but if you deploy with Google App Engine and rightfully expect their boundary proxies to compress your data for you, then you have to make sure your clients's
User-Agent is either whitelisted or contains the magic four bytes
gzip. (So if your client happens to be named "tagzipper" for example, Google will gzip for you.)
In their defense this is documented behavior, but it doesn't make it any less quirky. As it turns out, we hit the billed quota we set because our clients aren't including "gzip" in their UA strings... D'oh.
Might be good to know!