Using toFuture in Spring webClient when using virtual threads

Question

I've been trying to implement an orchestration system using VirtualThreads.

So far, I understand the underlying problems that I may face, and the one that really concerns me is "pinning."

I've decided to implement some classes using JFR to have a very detailed trace and metrics about where and when a virtual thread is being pinned to a carrier thread.

During this process, I found that the classic Apache HttpClient 4.x (the one used by all the projects I work on) pins virtual threads to carrier threads when the connection pool is full. This happens because of an NIO call that blocks the VT until a connection becomes available in the underlying connection pool. This kind of pinning lasts for 200-400 ms, depending on the pool, which I think is unacceptable. This situation motivated me to migrate to Apache HttpClient 5, which I found to be compatible with VTs.

After some further testing, I noticed that other portions of the code were also pinning threads, particularly when calling the execute() method of the HttpClient.

After some back and forth, I came up with a solution that I'd like to share to discuss whether it is a good approach or if it might lead to other problems.

I decided to create a WebClient (instead of a RestClient ) from WebFlux:

return WebClient.builder()
        .baseUrl(baseUrl)
        .filter(new CustomWebClientFilter(this.clientId).toExchangeFilterFunction())
        .codecs(configurer -> {
          configurer.defaultCodecs().jackson2JsonEncoder(new Jackson2JsonEncoder(objectMapper));
          configurer.defaultCodecs().jackson2JsonDecoder(new Jackson2JsonDecoder(objectMapper));
        })
        .build();

Then, when using this WebClient, I use toFuture() to avoid reactive programming throughout my entire application. Here's how I implemented it:

public XsDiscountResponse getById(String socialId, String company) {
  try {
    CompletableFuture<XsDiscountResponse> future = this.connector
            .get()
            .uri(uriBuilder -> uriBuilder
                    .path(GET_BY_ID_PATH)
                    .queryParam("company_id", company)
                    .build(socialId))
            .accept(APPLICATION_JSON)
            .retrieve()
            .onStatus(
                    HttpStatusCode::isError,
                    response -> response.bodyToMono(String.class)
                            .flatMap(body -> Mono.error(
                                    new ConnectorException(
                                            "Error: " + response.statusCode() + ", Body: " + body))))
            .bodyToMono(XsDiscountResponse.class)
            .toFuture(); // Convert Mono to CompletableFuture

    return future.get(); // Compatible with Loom for non-pinning waits
  } catch (Exception ex) {
    throw new ConnectorException("Error occurred during the request", ex);
  }
}

With this solution, I still handle my external dependencies in a blocking style, but I delegate the scheduling of threads to the VTs in my JVM using Future.

If I use the block() method from Mono, I understand that it will cause pinning.

Is there anything in this approach that is incorrect? Am I missing something important about how this blocking/async code might behave in a real-world application?

I will continue testing and monitoring pinned threads and response times to ensure everything works as expected. However, since VTs are relatively "new" and I haven't used asynchronous WebClient before, I'm not completely sure if this approach is correct.

Yes, this should work. But 1) using WebClient you don't avoid Reactive paradigm, as it uses Event Loop which is essential if not major part of Reactive, 2) in a way you configure your WebClient, which is in fact a default way in terms of threading, it will use non-virtual Event Loop threads. Are you OK with that? The getById method you brought won't benefit from virtual threading whatsoever because the I/O waiting will take place on these non-virtual Event Loop threads. — igor.zh
– igor.zh, Commented Feb 3 at 14:19
Hi! Thanks for your comment. There is something that catch my eye. When you say "it will use non-virtual Event Loop threads" are you saying that the event loop thread that uses spring-webflux is a plataform thread that spring handle by itself? I was thinking that the event loop was going to take place inside the VT in which im executing the api call. — Martin Locurcio
– Martin Locurcio, Commented Feb 4 at 15:51
If I understand your question correctly, yes, the threads participating in Event Loop are handled by spring-webflux itself. In fact, under the hood, not spring itself, it looks like Netty is in the game, but anyway not by the virtual thread executor (or similar machinery) you use in your application to enable virtual threads. By the way, how do you enable them? By setting spring.threads.virtual.enabled to true? — igor.zh
– igor.zh, Commented Feb 4 at 17:01
In fact, this Event Loop is started once and will be used by any WebClient instance in any your thread, so it is kind of "detached" thing. Will be used, I guess, for any incoming requests, if you are expecting them. — igor.zh
– igor.zh, Commented Feb 4 at 17:11
@igor.zh good! I've doing some research and I decided to stick with a RestClient instead of a WebClient to avoid webflux completely. I've enabled VTs with that property and all of the request of my application are handled by VTs. What i've trying to accomplish is to use VTs as well for blocking operations consuming external services. In this process I found out that restClient (spring) with apachehttpclient5 causes some "pinning" in the VTs, but this pinning is just for 1ms and it seems that is not really a problem for my use case. — Martin Locurcio
– Martin Locurcio, Commented Feb 5 at 13:45

igor.zh · Accepted Answer · 2025-02-05 17:18:12Z

Yes, converting a WebFlux/Reactor's Mono to a Future and subsequent invocation of get will work, but the pinning is avoided not because you don't use block method but because the actual waiting for I/O operation happens on Reactor's Event Loop. In a way you configure your WebClient instance, this Event Loop will use non-virtual, platform threads, you could see them in a thread dump under the names reactor-http-nio-*.

Thus, you are not avoiding Reactive Programming as you intended, but instead are using its cornerstone feature - Event Loop with its limited amount of threads. I don't mean to say that it is necessarily bad or should be avoided in the presence of virtual threads, it is very complex question and a solution depends on many factors, some of them discussed in a thread Do Java 21 virtual threads address the main reason to switch to reactive single-thread frameworks?, but the usage of WebClient in the method getById defeats the whole purpose of virtual threads - the waiting for I/O operation happens on platform thread. Again, I don't mean to say that the usage of virtual threads is useless for your entire application, I only mean your getById method.

In fact, it might be possible to reconfigure WebClient to use virtual threads as discussed in the thread Configuring Spring WebFlux' WebClient to use a custom thread pool, but it does not look like virtual thread paradigm fits very well to Reactive paradigm as the latter suggests limited amount of unending, looping threads.

As far as thread pinning is concerned, WebClient looks to me as susceptible to pinning as Apache Http Client because sun.nio.ch.SelectorImpl the former's uses waits in native method WEPoll.wait at the same time holding an intrinsic synchronization monitor on publicSelectedKeys, i.e. inside of synchronized block, but, like I said above, pinning is not happening there because WebClient uses platform threads. It is also worth to note that in the latest versions of Java virtual thread pinning is expected to be eliminated, see for the details JEP 491: Synchronize Virtual Threads without Pinning.

Collectives™ on Stack Overflow

Using toFuture in Spring webClient when using virtual threads

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related