Virtual vs Platform Threads When blocking operations return too fast

Contents

final update

See this post for the most accurate results:

Virtual vs platform threads when making API calls

Update

After some feedback, I ran some new tests using code that is mentioned in JEP 444: Virtual Threads. which is this one:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
void handle(Request request, Response response) {
    var url1 = ...
    var url1 = ...
    var url2 = ...
 
    try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
        var future1 = executor.submit(() -> fetchURL(url1));
        var future2 = executor.submit(() -> fetchURL(url2));
        response.send(future1.get() + future2.get());
    } catch (ExecutionException | InterruptedException e) {
        response.fail(e);
    }
}
 
String fetchURL(URL url) throws IOException {
    try (var in = url.openStream()) {
        return new String(in.readAllBytes(), StandardCharsets.UTF_8);
    }
}

This code is a good start, but I needed to alter it a bit to look more like the use cases I have. The application I am developing has 20_000 tasks it needs to run so the more I can do each second the better performance I get.

The previous example has one parent thread and starts 2 virtual threads doing their own request each time the handle(…) method is called. In my use-case I have 20_000+ tasks that each do three get requests to end-points in a Spring application. To simulate requests that take more time I added a delay that can be changed by passing a path variable to the endpoint.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
@GetMapping("/delay/{t}")
String youChoseTheDelay(@PathVariable int t){

    try {
        Thread.sleep(t);
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }
    
    return generateHtmlPageWithUrls(100, "crawl/delay/");
}

The code to test the performance was this class. Switching from one executor to another was done by making it a comment. This is the class I was using:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
public class PageDownloader {

    public static void main(String[] args) {

        int totalRuns = 20;

        for (int s = 0; s < totalRuns; s++) {

            long startTime = System.currentTimeMillis();

            //try (var ex = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors()){
            try (var ex = Executors.newVirtualThreadPerTaskExecutor()) {
                IntStream.range(0, 20_000).forEach(i ->
                {
                    ex.submit(() -> {
                        try {
                            String s1 = fetchURL(URI.create("http://192.168.1.159:8080/v1/crawl/delay/0").toURL());
                            String s2 = fetchURL(URI.create("http://192.168.1.159:8080/v1/crawl/delay/0").toURL());
                            String s3 = fetchURL(URI.create("http://192.168.1.159:8080/v1/crawl/delay/0").toURL());

                            if (!s1.startsWith("<html>") || !s2.startsWith("<html>") || !s3.startsWith("<html>")) { // small check is responses are oke
                                System.out.println(i + " lenght is: " + s1.length());
                                System.out.println(i + " lenght is: " + s2.length());
                                System.out.println(i + " lenght is: " + s3.length());
                            }

                        } catch (IOException e) {
                            throw new RuntimeException(e);
                        }
                    });
                });

            }
            measureTime(startTime, 20_000);
        }

    }


    static String fetchURL(URL url) throws IOException {
        try (var in = url.openStream()) {
            return new String(in.readAllBytes(), StandardCharsets.UTF_8);
        }
    }

    private static void measureTime(long startTime, int visited) {
        long endTime = System.currentTimeMillis();
        long totalTime = endTime - startTime;

        double totalTimeInSeconds = totalTime / 1000.0;

        double throughput = visited / totalTimeInSeconds;
        System.out.println((int) Math.round(throughput));
    }

}

Almost everything you see is wrapped in a for-loop that runs 20 times to get some data on what the throughput was of the application.

Results

So all this testing gave me the following results. On average the Spring application end-point returned a response within 5ms, so every task has to wait 3 times ~5ms. This is without the extra delay that can be added through the Thread.sleep().

JDK	extra delay	Executor	Throughput avg 20 runs
21	0	newVirtualThreadPerTaskExecutor	3736
21	0	newFixedThreadPool	4172
21	1	newVirtualThreadPerTaskExecutor	3482
21	1	newFixedThreadPool	3287
21	5	newVirtualThreadPerTaskExecutor	3554
21	5	newFixedThreadPool	1667
21	10	newVirtualThreadPerTaskExecutor	3587
21	10	newFixedThreadPool	826
23	0	newVirtualThreadPerTaskExecutor	3323
23	0	newFixedThreadPool	4149
23	1	newVirtualThreadPerTaskExecutor	3479
23	1	newFixedThreadPool	3286

The results show that if the task spends minimal time in a blocking state it is a better fit for platform threads. When tasks spend a longer time in a blocking state they are a good fit for Virtual Threads

All in all, for this application running on this machine I think I can safely say that if a task that I have spends less than 3 times 5ms in a blocking state it should run a platform thread instead of a virtual thread. In all other cases, the virtual thread outperforms the platform thread.

Intro

I wanted to see if Virtual threads would be a great fit for a web scraper. Every single web page would be handled by a single Virtual Thread. Web requests are blocking so at first sight this should be a great fit to use virtual threads. Turns out they are a good fit for certain requests but not all.

Background

Virtual threads introduce a layer of abstraction on top of traditional platform threads. They run on carrier threads which are essentially just platform threads. The key difference is that when a virtual thread encounters a blocking operation (like waiting for a response from a web server), it's unmounted from the carrier thread. This allows the carrier thread to pick up other Virtual Threads, improving the utilization of the hardware.

Testing setup

the setup exists of a web scraper written in Java 21 using Virtual threads and a web server using Spring. The web server provides two endpoints both returning a list of URLs. One returns the list of URLs immediately and the other returns them with a delay between 10ms and 200ms. For the most realistic result, I ran the two applications on different machines.

The scraper used 16 carrier threads for the virtual threads. The runs with platform threads were done with a pool of 16 platform threads.

Performance for Blocking I/O

Most places on the internet (also this blog) tell you that Virtual Threads are great at blocking I/O operations and looking at the numbers that seems to be very true. The following stats are from running the web scraper with either Virtual threads or Platform threads using the delay endpoint of the webserver.

As expected, virtual threads shine when dealing with long blocking operations, as seen in the graph above. The results from five runs show virtual threads scraping over 1400 pages per second, compared to platform threads’ 270 pages per second. This clearly shows the advantage of virtual threads for tasks involving I/O operations that take a long time to complete.

UPDATE: I ran the test for different JDK versions to see how this impacts the results.

Performance for Short Blocking I/O

So how do virtual threads perform when the blocking operation doesn't take a long time? To get these results I ran the web scraper five times and using the endpoint without the random delay.

Here, the graph reveals that platform threads outperform virtual threads by around 200 pages per second. This is because of the scheduler used to schedule the virtual threads.

UPDATE: I ran the test for different JDK versions to see how this impacts the results. As you can see the version of the JDK matters a lot when doing performance tests. Something seems to have changed between JDK 22 and 23 that negatively impacts the performance of this use case.

Conclusion

Virtual threads are not a silver bullet. While they excel at handling long blocking I/O operations, sometimes the scheduler negate the benefits for tasks with minimal time spent in a blocking state. To make a good choice for your application I recommend doing performance benchmarks to see if Virtual thread are indeed improving your applications’ performance.

If you want to know more about Virtual threads, I recommend reading JEP 444.

Virtual threads