is there any way to handle HTTP/2 Goaway received IOException in HttpClient java?

Question

I am making API calls in the application, At some point randomly it throws java.io.IOException: /149.222.1.1:553232: GOAWAY received, using Java 11 environment. anyway to resolve this exception rather than moving to Http/1.1 or other HTTP client libraries?

here is the stack trace

"com.abcd.common.RequestProcessor:53 getRequest: java.io.IOException: /149.222.1.1:553232: GOAWAY received
    at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:565)
    at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:119)
    at com.abcd.common.RequestProcessor.getRequest(RequestProcessor.java:49)
    at com.abcd.common.util.SendApiRequestUtil.sendApiRequestToCMS(SendApiRequestUtil.java:97)
    at com.abcd.controller.ContactsController.getContacts(ContactsController.java:61)
    at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:878)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:792)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:645)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651)
    at com.abcd.filter.LogFilter.doFilter(LogFilter.java:26)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
    at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:141)
    at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:82)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
    at org.eclipse.jetty.server.Server.handle(Server.java:501)
    at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
    at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.IOException: /169.254.8.1:55302: GOAWAY received
    at java.net.http/jdk.internal.net.http.Http2Connection.handleGoAway(Http2Connection.java:985)
    at java.net.http/jdk.internal.net.http.Http2Connection.handleConnectionFrame(Http2Connection.java:853)
    at java.net.http/jdk.internal.net.http.Http2Connection.processFrame(Http2Connection.java:724)
    at java.net.http/jdk.internal.net.http.frame.FramesDecoder.decode(FramesDecoder.java:155)
    at java.net.http/jdk.internal.net.http.Http2Connection$FramesController.processReceivedData(Http2Connection.java:232)
    at java.net.http/jdk.internal.net.http.Http2Connection.asyncReceive(Http2Connection.java:649)
    at java.net.http/jdk.internal.net.http.Http2Connection$Http2TubeSubscriber.processQueue(Http2Connection.java:1275)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
    at java.net.http/jdk.internal.net.http.Http2Connection$Http2TubeSubscriber.runOrSchedule(Http2Connection.java:1293)
    at java.net.http/jdk.internal.net.http.Http2Connection$Http2TubeSubscriber.onNext(Http2Connection.java:1319)
    at java.net.http/jdk.internal.net.http.Http2Connection$Http2TubeSubscriber.onNext(Http2Connection.java:1253)
    at java.net.http/jdk.internal.net.http.common.SSLTube$DelegateWrapper.onNext(SSLTube.java:202)
    at java.net.http/jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper.onNext(SSLTube.java:484)
    at java.net.http/jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper.onNext(SSLTube.java:287)
    at java.net.http/jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher.run1(SubscriberWrapper.java:318)
    at java.net.http/jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher.run(SubscriberWrapper.java:261)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:271)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:224)
    at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.outgoing(SubscriberWrapper.java:234)
    at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.outgoing(SubscriberWrapper.java:200)
    at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader.processData(SSLFlowDelegate.java:403)
    at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader$ReaderDownstreamPusher.run(SSLFlowDelegate.java:264)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SynchronizedRestartableTask.run(SequentialScheduler.java:175)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:147)
    at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:198)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

if the server decides to break the connection, switching client libraries won't help in any way. — eis
– eis, Commented Apr 10, 2021 at 5:39

Jan Tosovsky · Accepted Answer · 2021-11-02 22:09:22Z

This is something that your code will just have to deal with.

A GOAWAY frame has been sent by the HTTP 2.0 server that your client is talking to. That means that the server has decided to shut down the connection for some reason. The only thing that the client side can do is tell the caller via an IOException.

This is analogous to an HTTP 1.x server doing a TCP reset ... or just disconnecting. These will also result in a client-side IOException.

So, the solution is to catch the IOException and handle it; e.g. by reporting to the user that the request has failed, or retrying the HTTP request using a fresh connection.

(If you retry, you need some strategy to avoid your client code repeating a request that is never going to succeed. Indeed, ill-advised retrying can actually make things worse, if the reason that the server disconnected is that it is overloaded.)

I doubt that switching to HTTP 1.1 would help. I would expect that you would continue to get "random" IOExceptions with a different message ... from the server using HTTP 1.1.

There were a couple of JDK bugs related to GOAWAY handling in HTTPClient, but they were marked as resolved in 2018. You can use this query to find them:

https://bugs.openjdk.java.net/browse/JDK-8198947?jql=text%20~%20%22httpclient%20goaway%22

I don't think they are relevant if you are using the latest version of JDK 11.

honeybadger · Accepted Answer · 2022-09-14 13:18:11Z

5

try adding ".version(HttpClient.Version.HTTP_1_1)" in your http request

answered Sep 14, 2022 at 13:18

honeybadger

631 silver badge1 bronze badge

3 Comments

I Stevenson Over a year ago

But this does not fit with in the OP's requirements of "anyway to resolve this exception rather than moving to Http/1.1".

Community Over a year ago

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

julaine Over a year ago

and on what object would I do that?

Rafael · Accepted Answer · 2025-03-27 13:22:00Z

3

This is an open bug on JDK11 and 17
https://bugs.openjdk.org/browse/JDK-8335181

You can use a different library like okhttp or others.

answered Mar 27 at 13:22

Rafael

2,7362 gold badges36 silver badges65 bronze badges

Comments

Lubo · Accepted Answer · 2022-04-29 08:55:42Z

1

I believe that given GOAWAY is send from server to "idle connection after being idle too long". Generally, you can configure to not log such exception if they occur in client holding connection on background.

In your case I see, that exception is thrown from your code:

com.abcd.common.RequestProcessor.getRequest(RequestProcessor.java:49)

This is not case described at the beginning of my answer.

What is "random point in time"? Is it during http call? Does server wait for longer period of time for your input to be sent?

PS: There is also great chance, that after your 1000th request using single TCP connection, server does close connection. It is a default value for nginx at least. See https://www.androidbugfix.com/2022/03/how-to-handle-http2-goaway-with.html for more info.

edited Apr 29, 2022 at 8:55

answered Apr 29, 2022 at 8:26

Lubo

1,86716 silver badges41 bronze badges

1 Comment

helvete Over a year ago

Original of the linked resource is this QA

ddc · Accepted Answer · 2024-01-17 12:09:21Z

We encountered this problem with our application. I tested it using three separate server instances (TEST1, TEST2, and Live).

Only the first two environments (TEST1 and TEST2) cause our programme to exhibit the behaviour described above; in the third environment (Live), however, everything functions as it should and we are not receiving the 'GOAWAY received' error.

As a temporary fix, I downgraded the HTTP Client version to 1.1, which resolved the problem in TEST1 and TEST2.

We haven't found the solution yet and are continuously looking into it because different circumstances create varied behaviours.

This conversation on github was really beneficial for me to move on.

https://github.com/OpenLiberty/open-liberty/issues/25279

Java version we are using is - openjdk version "11.0.21" 2023-10-17 LTS

We would be grateful for any suggestions.

Collectives™ on Stack Overflow

is there any way to handle HTTP/2 Goaway received IOException in HttpClient java?

5 Answers 5

Comments

3 Comments

Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

3 Comments

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related