Go http client doesn't automatically dechunk body

Question

I'm streaming http from Go and the server responds with "Transfer-Encoding: chunked" as expected. And I've been told that the http client in Go shall automatically dechunk the body from the http response, removing the \r\n. But in my case it isn't removed automatically, so I have to use a ChunkedReader to read the bodies.

Any idea why golang doesn't dechunk my body automatically?

EDIT: Here is the http request:

var transport = http.Transport{
  Proxy:                  nil,
  ExpectContinueTimeout:  0,
  MaxResponseHeaderBytes: 16384}

var httpClient = http.Client{
  Transport: &transport,
  CheckRedirect: func(req *http.Request, via []*http.Request) error {
    return http.ErrUseLastResponse
}}

bodyReader, bodyWriter := io.Pipe()
req, _ := http.NewRequest("GET", "http://x.x.x.x/stream", bodyReader)
response, err := httpClient.Do(req)

buffer := make([]byte, 2 << 15)
n, readErr = response.Body.Read(buffer)   <-- should be dechunked body

The data read into the buffer is not dechunked. Any idea why?

What does "dechunk the body from the http response, removing the \r\n" mean? Dechunking is not (at least not directly) related to removing Windows lineendings? — Volker
– Volker, Commented Sep 12, 2019 at 9:03
In any case, if you want help debugging your code, you must include all relevant code in the question. Please update your question with a Minimal, Reproducible Example. — Jonathan Hall
– Jonathan Hall, Commented Sep 12, 2019 at 9:27
In golang.org/src/net/http/response.go at row 66, it says "The Body is automatically dechunked if the server replied with a "chunked" Transfer-Encoding.".. this does not happen. Why? That is my question. — Lalle
– Lalle, Commented Sep 12, 2019 at 10:10
As already explained, until we can see your code, there's no possible way to tell you why it does anything. — Jonathan Hall
– Jonathan Hall, Commented Sep 12, 2019 at 10:53

Lalle · Accepted Answer · 2019-09-18 11:27:21Z

1

I figured out why the body is not dechunked automatically. It's because the HTTP response was HTTP/1.0. In which case golang ignores the transfer encoding header.

https://github.com/golang/go/issues/12785

answered Sep 18, 2019 at 11:27

Lalle

74014 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Go http client doesn't automatically dechunk body

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related