how to catch "http: server closed idle connection" error

Question

In my go application I am getting the following error: "http: server closed idle connection". I would like to catch it and retry my http connection if it's encountered.

I found that this error comes from the "net/http" package and furthermore from the transport implementation. In particular it's defined here I get it wrapped in url.Error but this is all I was able to find out. Do you know how I can actually catch this error?

Edit: I am using elastic search client, which in turn is using the net/http. From the client I get the above mentioned error and would like to retry my elastic search request as being transient. For now the way I am catching transient errors is:

if urlErr, ok := err.(*url.Error); ok && (urlErr.Temporary() || urlErr.Err == io.EOF) {
    return retryRequest()

}

according to the comment, you should not get this error. error values for debugging and testing, not seen by users. — zzn
– zzn, Commented Mar 17, 2017 at 1:50
are you writing a server or client? can you give an example of what you're trying to do? — MahlerFive
– MahlerFive, Commented Mar 17, 2017 at 2:57
Are you using a current version of Go? Like zzn said, you shouldn't see that error because it should just trigger the client to retry the request. — Mr_Pink
– Mr_Pink, Commented Mar 17, 2017 at 19:03
@JimB this is what I thought. I am using current version 1.7.1. Do you think I should file a bug? — Ela
– Ela, Commented Mar 17, 2017 at 20:29
@Ela: the current version is go1.8, and you need an example of how to reproduce the error if you see this in go1.8 (and I would check master as well). — Mr_Pink
– Mr_Pink, Commented Mar 17, 2017 at 20:32

Gray Fox · Accepted Answer · 2017-09-21 16:31:50Z

5

Comments in net/http/transport.go say this (located here):

if err == errServerClosedIdle {
    // The server replied with io.EOF while we were trying to
    // read the response. Probably an unfortunately keep-alive
    // timeout, just as the client was writing a request.
    return true
}

Go will attempt to retry the request if it finds a non-nil http.Request.GetBody (found here), so I think it is expected to retry the request (or provide a GetBody function).

As for the leading error cause, you might want to check server's keep-alive functionality, my guess is that server is sending TCP connection reset (interpreted as io.EOF) prematurely. You might want to try disabling keep-alives and see if that changes anything.

answered Sep 21, 2017 at 16:31

Gray Fox

6731 gold badge8 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ela Over a year ago

thanks for your answer. I posted this question a long time ago and don't have enough context to know if this is a satisfactory answer.

Gray Fox Over a year ago

That's fine, I myself experienced the same issue and was hunting it down for a long time (and stumbled upon this question multiple times), so I decided to at least document it.

kimbo · Accepted Answer · 2020-06-09 22:15:16Z

As Gray Fox mentioned, Go will usually retry these requests. However, this is not the case for requests that aren't "idempotent". I was running into a few of these errors on POST requests, and after searching around I found this Github issue that says:

The HTTP Transport would normally retry those requests, but because they're POST requests they're not idempotent and Go conservatively assumes it might be unsafe to retry them.

Later in the issue someone mentioned that many banking APIs use some variant of the Idempotent-Key header to make them "safe". Stripe is a company that uses that header, and this is what their API docs say about it:

The API supports idempotency for safely retrying requests without accidentally performing the same operation twice.

So there you go, that's what the header is for.

In short, if you want Go's HTTP Transport to retry POST requests (and other non-idempotent requests), you need to include either the Idempotency-Key or X-Idempotency-Key header. That doesn't necessarily make those requests idempotent, but it will make them retry-able from net/http's perspective.

Links:

Source code check for the Idempotency-Key headers - https://github.com/golang/go/blob/2cd2ff6f564dce5be0c4fb7f06338ff7af3fc9a9/src/net/http/request.go#L1401)
Github issue where I found this information: https://github.com/golang/go/issues/19943

Collectives™ on Stack Overflow

how to catch "http: server closed idle connection" error

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related