How to parse http headers in Go

Question

I have http response headers shipped in logs from elsewhere. In my log file I have things like :-

Date: Fri, 21 Mar 2014 06:45:15 GMT\r\nContent-Encoding: gzip\r\nLast-Modified: Tue, 20 Aug 2013 15:45:41 GMT\r\nServer: nginx/0.8.54\r\nAge: 18884\r\nVary: Accept-Encoding\r\nContent-Type: text/html\r\nCache-Control: max-age=864000, public\r\nX-UA-Compatible: IE=Edge,chrome=1\r\nTiming-Allow-Origin: *\r\nContent-Length: 14888\r\nExpires: Mon, 31 Mar 2014 06:45:15 GMT\r\n

Given the above as string, how go I parse it into Header object as described in net/http . One way would be to split the string myself and map the key, values... But I am looking to avoid doing that by hand and use the standard (or well maintained 3rd party) library to parse it... Any pointers?

@dystroy it doesn't look like a duplicate to me, could you please elaborate ? — SirDarius
– SirDarius, Commented Mar 21, 2014 at 14:48
It's pretty trivial to parse this. It's just a string of key/value pairs each separated by line breaks. You may want to roll up your sleeves on this one. — Ralph Caraveo
– Ralph Caraveo, Commented Mar 21, 2014 at 15:11
@ralph-caraveo yes its trivial, but im lazy. I like to own as less code as possible... also the accepted answer also brings in things like canonicalization, etc for free. — sajal
– sajal, Commented Mar 21, 2014 at 18:19

Zombo · Accepted Answer · 2021-01-02 18:23:39Z

The builtin parser is in textproto. You can either use this directly, or add a fake HTTP request header and use ReadRequest in the http package. Either way you need to wrap your data into a bufio.Reader, here I'm just assuming we're starting with a string.

With textproto:

logEntry := "Content-Encoding: gzip\r\nLast-Modified: Tue, 20 Aug 2013 15:45:41 GMT\r\nServer: nginx/0.8.54\r\nAge: 18884\r\nVary: Accept-Encoding\r\nContent-Type: text/html\r\nCache-Control: max-age=864000, public\r\nX-UA-Compatible: IE=Edge,chrome=1\r\nTiming-Allow-Origin: *\r\nContent-Length: 14888\r\nExpires: Mon, 31 Mar 2014 06:45:15 GMT\r\n"

// don't forget to make certain the headers end with a second "\r\n"
reader := bufio.NewReader(strings.NewReader(logEntry + "\r\n"))
tp := textproto.NewReader(reader)

mimeHeader, err := tp.ReadMIMEHeader()
if err != nil {
   log.Fatal(err)
}

// http.Header and textproto.MIMEHeader are both just a map[string][]string
httpHeader := http.Header(mimeHeader)
log.Println(httpHeader)

and with http.ReadRequest:

logEntry := "Content-Encoding: gzip\r\nLast-Modified: Tue, 20 Aug 2013 15:45:41 GMT\r\nServer: nginx/0.8.54\r\nAge: 18884\r\nVary: Accept-Encoding\r\nContent-Type: text/html\r\nCache-Control: max-age=864000, public\r\nX-UA-Compatible: IE=Edge,chrome=1\r\nTiming-Allow-Origin: *\r\nContent-Length: 14888\r\nExpires: Mon, 31 Mar 2014 06:45:15 GMT\r\n"

// we need to make sure to add a fake HTTP header here to make a valid request.
reader := bufio.NewReader(strings.NewReader("GET / HTTP/1.1\r\n" + logEntry + "\r\n"))

logReq, err := http.ReadRequest(reader)
if err != nil {
   log.Fatal(err)
}

log.Println(logReq.Header)

https://golang.org/pkg/net/textproto

This works for email also since net/mail uses textproto for mime parsing.

Collectives™ on Stack Overflow

How to parse http headers in Go

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related