I want to send large amount of json over http to sever.
If I use "Content-Encoding":"GZIP" in my httpClient, does it automatically convert the request body to compressed format?
2 Answers
No, the RFC 7231 describes content encoding. If you are sending Content-Encoding you need to make sure that the content is in that encoding.
If you send Content-Encoding: gzip and the message in plain text you will (quite rightly) receive an HTTP 400. The body of a gzip message will always start with 0x1f 0x8b and if the server does not find that int he POST request it is right to complain.
Another reason for this is that you need an appropriate Content-Length header. This will not be the length of the original JSON, it must be the length (in bytes) of the gzipped JSON.
You need to perform the gzip of the JSON before sending anything since you need to know what to place in Content-Length beforehand.
Extra note: If the JSON is that huge (e.g. several gigabytes) you probably will need Transfer-Encoding: chunked, which comes with its own complications. (You do not send Content-Length but add the length of the chuck to the body itself.)
4 Comments
Julian Reschke
a) RFC 2295 is totally irrelevant; see RFC 7231 instead. b) I have no idea why you think T-E is relevant here...
grochmal
@JulianReschke - a) You're right, updated. b) T-E is allowed in a POST (as far as I am aware), it is an option for OP in case his JSON is huge (not strictly needed for the answer, but may be useful).
user902997
When the compressed content reaches server with a header "Content-Encoding":"GZIP", does the server automatically decompress it?
grochmal
@user902997 - Yes and no. The server must read the entire message, check
Content-Encoding and then decompress the message. On the other hand: Apache, Cherokee, Nginx, (and likely almost all others) will do the transparently and give an already decompressed payload to the actual application (say PHP or Rails).If it automatically does this, is 100% dependent on which http client you are using and if they implemented it that way. Usually setting a header will not automatically encode it, at least in the clients I regularly use.
