How to specify the remote IP dynamically when send HTTP(s) requests using net/http package

Question

Requirement: worker goroutines concurrently send HTTPS(S) requests to various IPs. The URL and server IP are specified by the "main" routine in the parameter list when creating workers.
For instance: there are 100 worker goroutines requests https://www.google.com/ from different server IPs. The first worker asks for it from 1.1.1.1, and the second one does exactly the same thing but a different server IP (say 2.2.2.2).
Current solution: assign one http.Client with customized dial function in http.Transport for each routine.

   go func(host, url string) {
        defer wg.Done()
        Client:=&http.Client{
        Transport: &http.Transport{
            DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
                n, e := net.Dial("tcp", host)
                fmt.Println(e)
                return n, e
            },
          },
       }

        resp, err := Client.Get(url)
        if err != nil {
            fmt.Println(err)
            return
        }

        defer resp.Body.Close()

        c, e := ioutil.ReadAll(resp.Body)
        if e != nil {
            return
        }
        fmt.Println(string(c))

    }(realHost, url)

according to the godoc, Clients and Transports are safe for concurrent use by multiple goroutines and for efficiency should only be created once and re-used.

Questions: So how can I fulfill my requirement by re-using the Client and transport?

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. — Community
– Community Bot, Commented May 30, 2022 at 5:11

Mises · Accepted Answer · 2022-05-29 17:27:59Z

0

Create a request and swap hosts in the request.

req, err := http.NewRequst(“GET”, url, nil)
req.Host = req.URL.Host // set host header
req.URL.Host = host     // set dial host

Use the default client and transport to execute the request.

 resp, err := http.DefaultClient.Do(req)

edited May 29, 2022 at 17:27

answered May 29, 2022 at 16:34

Mises

11 bronze badge

Sign up to request clarification or add additional context in comments.

2 Comments

Shane Over a year ago

Thanks, but 1) why do you do req.Host = req.URL.Host? I guess req.Host always equals to req.URL.Host by default. What makes them different? 2) I don't think this solution works with HTTPS, it fails to check SSL cert (certificate name does not match input).

Mises Over a year ago

(1) the host header is different from the dial address. (2) Yeah, this only works with http.

Collectives™ on Stack Overflow

How to specify the remote IP dynamically when send HTTP(s) requests using net/http package

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related