2

How to know the size of the chunk of HTTP response if transfer encoding is chunked.I am unable to get the logic. Please help me.And provide me some sample java code to get the size of chunk.I read in some books that size of each chunk is specified before the chunk itself.But using which logic can I get that. Please help me using java.

Thank you.

Improve this question

1 Answer 1

Reset to default
5

Not able to give ready to use code but Apache HttpClient library supports "everything" out of the box.

This is wikipedia example of chunked response. You don't know the exact byte size of data until body part is read. Please note size of each chunk is hexadecimal value, last 0 sized chunk is end of data. It too is terminated by 2-byte CRLF delimiter.

Transfer-Encoding: chunked\r\n
Content-Type: text/plain\r\n
\r\n
4\r\n
Wiki\r\n
5;extkey=extvalue\r\n
pedia\r\n
E\r\n
.in\r\n
\r\n
chunks.\r\n
0\r\n
AfterBodyHeader: some value\r\n
AfterBodyHeader2: any value\r\n
\r\n
  • read bytes until CRLF (\r\n or 0D0A in hex)
  • drop everything after ; delimiter or stop at trailing CRLF, some replys may add extensions at the line of chunk size. Ignore them unless you wait for known key-value pair.
  • convert hex to integer, it tells you num of data bytes in chunk
  • read x num of bytes and append to ByteArrayOutputBuffer
  • read trailing CRLF delimiter bytes and ignore it
  • if not 0 sized read again size+chunk data
  • some responses may add AfterHeaders they are like headers at the start of response
  • after 0 sized chunk an empty line (\r\n) indicates end of complete http response. If there is no AfterHeaders you should see bytes 0\r\n\r\n in the end. Please note regular chunks may contain empty lines so do not terminate parsing until 0-sized indicator.

This example uses \r\n placeholders to indicate 2-byte delimiter as specs require

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

thank you..but I found in one page that chunk_size = int(line.strip().split(';')[0], 16) I did not understand this line.They have given this logic to find chunk size.but what is [0] index and why 16 is placed.I did not understand this statement.please can u help me.
"5;extkey=extvalue".split(;) gives an array of two elements [0]="5", [1]="extkey=extvalue". int(arr[0],16) converts first string element to integer value, please note size of chunk is hexadecimal thats why an indicator of 16. If line does not have extension(;) split just returns one-sized array and strToInteger conversion still works.
the above logic i.e., int(arr[0],16) directly gives me the chunk size in bytes it seems.Am I correct??
int(arr[0],16) function converts hex string to integer value. Its the num of data bytes belonging to a next chunk. Yes 4 means "Wiki" bytes and 5 means "pedia" bytes. Please note \r\n terminator bytes at the end of each line and they are not included in chunk size. You just have to know there is those "hidden" two bytes on each line end. Size of "E" hex is 14 bytes ".in\r\n\r\nchunks." new lines within chunk data like any data bytes.
According to above example first I need to read 4\r\n and then convert 4 to integer. After conversion if it gives some value say X, then I have to read X bytes and append to ByteArrayOutputBuffer.that means this time I am reading "Wiki" right? and I should read \r\n which is there after "wiki" and ignore it.And again read 5\r\n and so on.Similar process continues until I get 0. Is this the way to get chunked data?Am I correct.Please give reply.thank you.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.