Specify timeout when tracing HTTP request in Go

Question

I know the usual method of specifying a timeout with HTTP requests by doing:

httpClient := http.Client{
    Timeout: time.Duration(5 * time.Second),
}

However, I can't seem to figure out how to do the same when tracing HTTP requests. Here is the piece of code I am working with:

func timeGet(url string) (httpTimingBreakDown, error) {
    req, _ := http.NewRequest("GET", url, nil)

    var start, connect, dns, tlsHandshake time.Time
    var timingData httpTimingBreakDown
    timingData.url = url

    trace := &httptrace.ClientTrace{
        TLSHandshakeStart:    func() { tlsHandshake = time.Now() },
        TLSHandshakeDone:     func(cs tls.ConnectionState, err error) { timingData.tls = time.Since(tlsHandshake) },
    }

    req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
    start = time.Now()

    http.DefaultTransport.(*http.Transport).ResponseHeaderTimeout = time.Second * 10 // hacky way, worked earlier but don't work anymore

    if _, err := http.DefaultTransport.RoundTrip(req); err != nil {
        fmt.Println(err)
        return timingData, err
    }

    timingData.total = time.Since(start)

    return timingData, nil
}

I am firing this function inside a goroutine. My sample data set is 100 urls. All goroutines fire, but eventually the program ends in 30+ secs as if the timeout is 30secs.

Earlier I made the same to work by using the hacky way of changing the default inside of it to 10 secs and anything that took too long, timed out and the program ended at 10.xxx secs but now its taking 30.xx secs.

What would be a proper way of specifying a timeout in this scenario?

Community · Accepted Answer · 2020-06-20 09:12:55Z

12

I know the usual method of specifying a timeout with HTTP requests by doing:
httpClient := http.Client{
    Timeout: time.Duration(5 * time.Second),
}

Actually, the preferred method is to use a context.Context on the request. The method you've used is just a short-cut suitable for simple use cases.

req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
    return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 5 * time.Second)
defer cancel()
req = req.WithContext(ctx)

And this method should work nicely for your situation as well.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Feb 21, 2018 at 8:27

Jonathan Hall

80.5k19 gold badges162 silver badges206 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Ashfame Over a year ago

Thank you! Any recommendations on keeping up with the latest recommended methods with newer Go versions of doing something? Often I land up on google results which are doing some hackery to accomplish something which is not needed anymore but I just don't know that. Makes sense?

Jonathan Hall Over a year ago

@Ashfame: Read the release notes for every new Go release.

Ashfame Over a year ago

@FlimzyThat's doable when I have gained some experience with the language but difficult when just starting out with a language.

Jonathan Hall Over a year ago

@Ashfame: The release notes are pretty short for every version. It shouldn't be very intimidating at all. Skip over the parts that don't interest you.

Jonathan Hall Over a year ago

You can start here with the Go 1.10 release notes which was released just recently.

|

Collectives™ on Stack Overflow

Specify timeout when tracing HTTP request in Go

1 Answer 1

8 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related