Golang parse raw HTTP/2 response

Question

I have a specific case where I need to parse HTTP/2 response output into Go's http.Response. Response itself has default structure:

$ curl --include https://google.com

HTTP/2 301
location: https://www.google.com/
content-type: text/html; charset=UTF-8
date: Mon, 15 Jun 2020 11:08:39 GMT
expires: Wed, 15 Jul 2020 11:08:39 GMT
cache-control: public, max-age=2592000
server: gws
content-length: 220

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="https://www.google.com/">here</A>.
</BODY></HTML>

Status and body itself doesn't matter, it's just an example.

http library has function ReadResponse(r *bufio.Reader, req *Request) (*Response, error) which does exactly what I need, but it fails parsing HTTP/2 with malformed HTTP version HTTP/2, however it works fine for HTTP/1.1 and HTTP/1.0. Also, after doing request with http.DefaultClient.Do() you can see that response's field Proto contains HTTP/2.0, which means that there is no problem with HTTP/2.

Any ideas how to parse this response?

http.ParseHTTPVersion func not parse HTTP/2.replace HTTP/2 to HTTP/1.1 or HTTP/2.0 . — eudore
– eudore, Commented Jun 15, 2020 at 12:19
it won't parse HTTP/2.0 as well, but HTTP/1.1 does the trick, won't simply replacing version break something in more complex responses? — zzell
– zzell, Commented Jun 15, 2020 at 12:31
Only match the body prefix is HTTP/2 , and then replace the first 7 bytes. — eudore
– eudore, Commented Jun 15, 2020 at 12:40
Yeah, I got you, I am just asking is it just a matter of first 7 bytes? Aren't different HTTP versions has different response format, parsing rules, etc? — zzell
– zzell, Commented Jun 15, 2020 at 12:49
FTR: this is not a "raw" HTTP/2 response. HTTP/2 uses a stateful binary protocol, so the raw response isn't human-readable and often not even self-contained. This is curl's rendering of an HTTP/2 response that immitates HTTP/1.x. — Peter
– Peter, Commented Jun 15, 2020 at 14:18

eudore · Accepted Answer · 2020-06-15 13:13:05Z

To summarize the discussion, the reason for the parse error is http.ParseHTTPVersion not parse HTTP/2, replace the prefix 7 bytes into HTTP/2.0 to fix it, or provide pr support for net/http to parse HTTP/2.

package main

import (
    "bufio"
    "bytes"
    "fmt"
    "io"
    "net/http"
)

func main() {
    fmt.Println("Hello, playground")
    req, _ := http.NewRequest("GET", "https://google.com", nil)
    {
        resp, err := http.ReadResponse(bufio.NewReader(bytes.NewBuffer(httpbody)), req)
        fmt.Println(resp, err)
        // err is malformed HTTP version "HTTP/2", because http.ParseHTTPVersion not parse "HTTP/2"
    }

    {
        body := bytes.NewBuffer(httpbody)
        prefix := make([]byte, 7)
        n, err := io.ReadFull(body, prefix)
        if err != nil {
            panic("handler err")
        }
        fmt.Println(n, err, string(prefix))
        if string(prefix[:n]) == "HTTP/2 " {
            // fix HTTP/2 proto
            resp, err := http.ReadResponse(bufio.NewReader(io.MultiReader(bytes.NewBufferString("HTTP/2.0 "), body)), req)
            fmt.Println(resp, err)
        } else {
            // other proto
            resp, err := http.ReadResponse(bufio.NewReader(io.MultiReader(bytes.NewBuffer(prefix[:n]), body)), req)
            fmt.Println(resp, err)
        }
    }
}

var httpbody = []byte(`HTTP/2 301
location: https://www.google.com/
content-type: text/html; charset=UTF-8
date: Mon, 15 Jun 2020 11:08:39 GMT
expires: Wed, 15 Jul 2020 11:08:39 GMT
cache-control: public, max-age=2592000
server: gws
content-length: 220

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="https://www.google.com/">here</A>.
</BODY></HTML>`)

output:

Hello, playground
<nil> malformed HTTP version "HTTP/2"
7 <nil> HTTP/2 
&{301 301 HTTP/2.0 2 0 map[Cache-Control:[public, max-age=2592000] Content-Length:[220] Content-Type:[text/html; charset=UTF-8] Date:[Mon, 15 Jun 2020 11:08:39 GMT] Expires:[Wed, 15 Jul 2020 11:08:39 GMT] Location:[https://www.google.com/] Server:[gws]] 0xc0000902c0 220 [] false false map[] 0xc0000f2000 <nil>} <nil>

Collectives™ on Stack Overflow

Golang parse raw HTTP/2 response

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related