Understanding Timeout Behavior in Java JDK HttpClient

I've been working with Apache CXF and noticed a change in how timeouts are handled in the newer versions (3.6.x and 4.x). These versions now use the new JDK HttpClient, and I’m trying to verify my understanding of its timeout behavior.

Background

Previously, in older CXF versions (3.5.x and earlier), the timeouts set via HTTPClientPolicy#setReceiveTimeout seemed to act like socket read timeouts. In the newer versions, the "Request Timeout" appears to be set using the JDK HttpClient, specifically with the rb.timeout(Duration.ofMillis(rtimeout)); call in HttpClientWrappedOutputStream#setProtocolHeaders. (https://docs.oracle.com/en/java/javase/11/docs/api/java.net.http/java/net/http/HttpRequest.Builder.html#timeout(java.time.Duration))

My Understanding

1. Socket Read Timeout (Old Behavior): This timeout starts counting down once the client has finished sending the request and is waiting for the server’s response.

2. Request Timeout (New Behavior with JDK HttpClient): This timeout starts as soon as the request begins, including the time taken to send the request data.

Due to this change, I believe that when a client sends a large file, it might trigger the "Request Timeout" during the file transfer itself, rather than while waiting for a response.

Observations

• When transferring large files, the client sometimes closes the connection unexpectedly (with a FIN package) while still transmitting data, which I suspect is due to the "Request Timeout".
• No explicit timeout exceptions are logged by the client. Instead, the server logs an "early EOF" error because the client closes the connection unexpectedly.
• Client logs include messages like java.io.IOException: Read end dead and other related stack traces, indicating an interruption in data transmission.

HttpClient Logging

Enabling trace logging for the JDK HttpClient shows logs such as:

INFORMATION: MISC: purgeTimeoutsAndReturnNextDeadline: handling 1 events, remaining 0, next deadline: 0
Mai 15, 2024 6:02:34 PM jdk.internal.net.http.HttpClientImpl purgeTimeoutsAndReturnNextDeadline
INFORMATION: MISC: Firing timer ResponseTimerEvent[TimeoutEvent[id=11, duration=PT9S, deadline=2024-05-15T16:02:34.622262100Z]]
Mai 15, 2024 6:02:34 PM jdk.internal.net.http.PlainHttpConnection close

Questions

1. Timeout Behavior: Is my understanding correct that the "Request Timeout" in the JDK HttpClient starts counting down when the request begins, including the time taken to send data?
2. Diagnosing Timeouts: Why doesn’t the CXF client report these timeouts as explicit exceptions, and how can clients diagnose such timeout issues more effectively?
3. Configuring Timeouts: Given that our application handles both small (a few KB) and large (up to 10 GB) files, how should clients configure their timeouts? Should the timeout be scaled with the file size, considering that it includes the file transfer time?

I would greatly appreciate any clarification or insights into these questions, as I’m concerned about potentially misunderstanding the new timeout behavior.

Thank you for your help!