How to capture IP address of server providing HTTP response

Question

Using Go's default HTTP client, I am unable to directly determine the IP address of the server that processed the request. For instance, when requesting example.com, what is the IP address that example.com resolved to at the time of the request?

import "net/http"

resp, err := http.Get("http://example.com/")

The resp object contains the resp.RemoteAddr property, but as detailed below it is not utilized during client operations.

 // RemoteAddr allows HTTP servers and other software to record
 // the network address that sent the request, usually for
 // logging. This field is not filled in by ReadRequest and
 // has no defined format. The HTTP server in this package
 // sets RemoteAddr to an "IP:port" address before invoking a
 // handler.
 // This field is ignored by the HTTP client.
 RemoteAddr string

Is there a straightforward way to accomplish this? My initial thought would be to:

Initiate a DNS lookup to remote domain
Create new http transport using returned A/AAAA records
Make the request
Set the RemoteAddr property on the response object

Is there a better way?

UPDATED - to use @flimzy's suggestion. This method stores the remote IP:PORT into the request.RemoteAddr property. I've also added support for multiple redirects so that each subsequent request has its RemoteAddr populated.

request, _ := http.NewRequest("GET", "http://www.google.com", nil)
client := &http.Client{
    Transport:&http.Transport{
        DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
            conn, err := net.Dial(network, addr)
            request.RemoteAddr = conn.RemoteAddr().String()
            return conn, err
        },
    },
    CheckRedirect: func(req *http.Request, via []*http.Request) error {
        request = req
        return nil
    },
}
resp, _ := client.Do(request)

At best you can get the IP address of a server that processed the response, at some point along the way. That could be a proxy, a load balancer, a WAF, a reverse proxy... what are you actually trying to accomplish here? Why do you need the IP? — Adrian
– Adrian, Commented Mar 20, 2018 at 13:27
@Adrian This last server in the response chain is fine. Creating an archive using the WARC standard has a metadata field for IP address. While there are warc writers for Golang, none of them (currently) support processing a response into WARC format. — InfamousLoop
– InfamousLoop, Commented Mar 20, 2018 at 14:20

Jonathan Hall · Accepted Answer · 2023-01-30 09:48:29Z

8

As far as I can tell, the only way to accomplish this with the standard library is with a custom http.Transport that records the remote IP address, for example in the DialContext function.

client := &http.Client{
    Transport: &http.Transport{
        DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
            var d net.Dialer
            conn, err := d.DialContext(ctx, network, addr)
            fmt.Printf("Remote IP: %s\n", conn.RemoteAddr())
            return conn, err
        },
    },
}
resp, _ := client.Get("http://www.google.com")

Tying the connection IP to the response is left as an exercise for the reader.

edited Jan 30, 2023 at 9:48

answered Mar 20, 2018 at 13:21

Jonathan Hall

80.5k19 gold badges162 silver badges206 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Will Over a year ago

What would be a good way to persist this in the response object?

Shindou-L · Accepted Answer · 2019-03-12 03:34:25Z

3

you also can build a request with trace:

request = request.WithContext(httptrace.WithClientTrace(request.Context(), &httptrace.ClientTrace{
        GotConn: func(connInfo httptrace.GotConnInfo) {
            fmt.Printf("target ip:%+v\n", connInfo.Conn.RemoteAddr().String())
        },
    }))
response, _:= client.Do(request)

answered Mar 12, 2019 at 3:34

Shindou-L

891 silver badge5 bronze badges

Comments

eis · Accepted Answer · 2019-07-22 09:20:12Z

0

httptrace form https://golang.org/pkg/net/http/httptrace/

req, err := http.NewRequest(method, url, strings.NewReader(body))
    trace := &httptrace.ClientTrace{
        GotConn: func(connInfo httptrace.GotConnInfo) {
            fmt.Printf("Got Conn: %+v\n", connInfo)
        },
        DNSDone: func(dnsInfo httptrace.DNSDoneInfo) {
            fmt.Printf("DNS Info: %+v\n", dnsInfo)
        },
    }
req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))

answered Jul 22, 2019 at 9:20

eis

11 bronze badge

Collectives™ on Stack Overflow

How to capture IP address of server providing HTTP response

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related