I/O Threading Models: Blocking, Non-Blocking & Virtual
Every server framework makes a fundamental choice about how threads relate to I/O operations. this topic covers the three models - thread-per-request, async NIO, and virtual threads - and why they lead frameworks like Tomcat, Netty, and Spring Boot to such different designs.
Learning objectives
- Explain why a thread blocked on I/O is unavailable for other work in the classic model
- Describe how an NIO Selector lets one thread monitor thousands of sockets
- Explain how virtual threads deliver blocking-style code with non-blocking scalability
- Distinguish cooperative from preemptive scheduling and know where each applies to Java threads
- Judge when reactive programming still earns its complexity versus virtual threads
Every network server does fundamentally the same thing: accept a connection, wait for data, do something with it, and send a response. The interesting design decision is what a thread does while it's waiting - because waiting is most of what a network server actually spends its time on. A request that takes 200ms end-to-end might spend 190ms of that blocked on a database call or a downstream HTTP request, and only 10ms doing actual CPU work.
Three fundamentally different answers to "what happens to the thread while it waits" produce three fundamentally different architectures, and understanding all three explains why Tomcat, Netty, WebFlux, and a modern Spring Boot app running on virtual threads look so different under the hood despite often solving the exact same business problem.
The first and oldest answer is: the thread just waits, parked by the operating system, doing nothing else until the data arrives. This is simple to write and simple to debug, but it means one OS thread is permanently tied up per in-flight request, and OS threads are not free - each one reserves roughly a megabyte of stack memory whether it's doing anything or not. The second answer is: don't dedicate a thread to a connection at all. Use a small number of threads, each one watching thousands of connections at once through an operating-system event-notification mechanism, and only touch a connection when it actually has data ready. This scales to enormous connection counts but demands a completely different programming style - callbacks or reactive chains instead of ordinary sequential code. The third and newest answer, virtual threads, tries to get both: ordinary blocking-looking code, but with the JVM itself managing the unmounting and remounting of a thread from I/O waits, so a small number of real OS threads can service a huge number of logical threads.
When a thread calls socket.read() or issues a database query, the operating system suspends it - moving it into the WAITING state - until the data arrives from the network or disk. The thread consumes its full stack allocation the entire time, but does zero useful work while waiting. For a web server juggling 1,000 concurrent database queries, that's 1,000 blocked threads holding roughly a gigabyte of stack memory between them, just sitting idle.
This is the traditional Tomcat model: one OS thread per HTTP request, blocking on whatever I/O that request needs along the way. It works well up to somewhere around 200 to 500 concurrent requests before memory pressure and context-switch overhead start becoming the limiting factor - and critically, the thread pool's size is the server's throughput ceiling. Once every thread in the pool is blocked on a slow downstream call, new requests simply queue, regardless of how much idle CPU the machine has.
The appeal of this model is that it's the easiest one to write and reason about: code reads top to bottom, a stack trace during an incident shows exactly where a request is stuck, and there's no callback indirection to untangle. Its ceiling is purely a resource-accounting problem - not enough OS threads to go around at high concurrency - which is precisely the constraint the other two models exist to remove.
💻 Code example
package concurrency.iomodels; import java.io.IOException; import java.net.ServerSocket; import java.net.SocketTimeoutException; public class BlockingIoDemo { public static void main(String[] args) { System.out.println("Demonstrating blocking I/O..."); // try-with-resources guarantees the OS-level listening socket is // released even on exception. Port 0 asks the OS for any free port. try (ServerSocket server = new ServerSocket(0)) { server.setSoTimeout(500); // Fail fast instead of hanging forever. System.out.println("Listening on port " + server.getLocalPort()); // This is the actual blocking call: the calling thread is parked // by the OS and does nothing else until a client connects or the // 500ms timeout elapses. server.accept(); } catch (SocketTimeoutException e) { // Expected: proves the thread really was blocked, doing nothing // useful, for the entire 500ms wait. System.out.println("Accept timed out - thread was blocked the whole time."); } catch (IOException e) { e.printStackTrace(); } } }
Java's java.nio package provides non-blocking I/O built around a different primitive: instead of one thread per connection, a single Selector thread monitors thousands of channels for readability or writability, using an operating-system event-notification mechanism - epoll on Linux, kqueue on macOS. When data actually arrives on one of those channels, the OS notifies the selector, which wakes up and processes only the channels that are actually ready. No thread ever sits waiting on a socket that has nothing to say.
The typical deployment is a thread-per-core model: a small number of threads, one per CPU core, each running its own selector and handling thousands of connections in a non-blocking way. Memory overhead drops from roughly a megabyte per connection to a few bytes per channel and buffer. This is how Netty, Nginx, Node.js, and Reactor Netty (the engine behind Spring WebFlux) all work - and it's extremely efficient, but the programming model pays for it. Non-blocking code turns sequential logic into nested callbacks or reactive operator chains: "fetch a user, then fetch their orders, then compute a total" becomes a chain of subscribers that's harder to debug, test, and reason about, and a stack trace during an incident shows only the event loop's machinery, not the business-logic chain that led there.
There's one rule that makes or breaks this model: every piece of code running inside the event loop must be non-blocking. If a handler does anything blocking at all - a synchronous database call, a blocking file read - it stalls the single thread that's also responsible for monitoring every other socket, and the entire server stops making progress until that one call finally returns.
💻 Code example
package concurrency.iomodels; import java.net.InetSocketAddress; import java.nio.channels.SelectionKey; import java.nio.channels.Selector; import java.nio.channels.ServerSocketChannel; public class NioSelectorDemo { public static void main(String[] args) throws Exception { System.out.println("Setting up an NIO Selector..."); // Selector is Closeable; try-with-resources releases the underlying // epoll/kqueue instance when this block exits. try (Selector selector = Selector.open()) { ServerSocketChannel serverChannel = ServerSocketChannel.open(); serverChannel.configureBlocking(false); // Required before registering with a Selector. serverChannel.bind(new InetSocketAddress(0)); serverChannel.register(selector, SelectionKey.OP_ACCEPT); System.out.println("Selector watching port " + serverChannel.socket().getLocalPort()); // A real event loop would now repeatedly call selector.select(), // which blocks only until at least one channel is ready, then // iterate selector.selectedKeys() handling each ready channel - // accepting new connections, reading available bytes - before // clearing the set and looping again. } } }
Virtual threads, introduced in Java 21, aim for the best of both earlier models. You write simple, ordinary blocking code, exactly like Model 1 - no callbacks, no reactive operators - but the JVM internally uses continuations, conceptually similar to Model 2's event-driven approach, to unmount a virtual thread the moment it blocks on I/O, freeing the underlying carrier OS thread to run a different virtual thread in the meantime.
The mechanism: the JDK's I/O operations - SocketInputStream.read() and friends - are implemented to detect when they're running on a virtual thread. Instead of blocking the OS thread the way they would on a platform thread, they park the virtual thread by saving its call stack to the heap, letting the carrier thread pick up other virtual threads' work. When the I/O completes, the saved continuation is resumed on whatever carrier thread happens to be available - not necessarily the one it started on.
The practical effect is dramatic at scale. Submitting 1,000 tasks that each sleep for 50 milliseconds to a fixed pool of 10 platform threads takes roughly (1000 / 10) x 50ms, about 5 seconds, because only 10 tasks can occupy a real OS thread at once and the rest queue. The exact same blocking code submitted to a virtual-thread-per-task executor finishes in close to 50 milliseconds total, because each sleep unmounts its carrier rather than occupying it - all 1,000 sleeps overlap almost entirely in wall-clock time.
💻 Code example
package concurrency.iomodels; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; public class VirtualThreadScaleDemo { public static void main(String[] args) throws Exception { // try-with-resources on an ExecutorService (since Java 19) blocks on // close() until every submitted task finishes. try (ExecutorService exec = Executors.newVirtualThreadPerTaskExecutor()) { // Spins up a brand-new, extremely cheap virtual thread for every // submitted task instead of reusing a small fixed pool - this is // what lets all 1000 tasks run essentially concurrently. for (int i = 0; i < 1000; i++) { exec.submit(() -> { // Stands in for a blocking DB/HTTP call. Because this runs // on a virtual thread, the sleep unmounts the carrier OS // thread instead of occupying it for the full duration. try { Thread.sleep(50); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } }); } } System.out.println("Finished processing 1000 concurrent tasks."); } }
Platform threads are scheduled preemptively by the operating system: the scheduler forcibly interrupts a running thread after its time slice - typically 1 to 10 milliseconds - expires, and switches to another thread, regardless of what the first thread was doing. Threads have no say in when they're preempted. This is what stops a CPU-bound thread from monopolizing the processor and starving everything else; the tradeoff is a real, if small, context-switch cost on every forced handoff.
Virtual threads are scheduled cooperatively on top of the JVM: a virtual thread runs until it voluntarily yields, which happens by calling a blocking operation - I/O, Thread.sleep(), waiting on a lock - or by explicitly yielding. The scheduler only ever switches at these yield points; there's no involuntary preemption of a virtual thread by the JVM scheduler. This is exactly why CPU-bound virtual threads get no benefit from being virtual: a tight compute loop with no blocking call inside it never yields, so it just occupies its carrier thread continuously, behaving like an ordinary platform thread the whole time.
Java runs a genuine hybrid: platform threads are preemptively scheduled by the OS, and virtual threads are cooperatively scheduled by the JVM on top of those same platform threads acting as carriers. A carrier thread is itself still preemptively scheduled by the OS - it's only the layer of virtual threads riding on top of it that behaves cooperatively. This is also why CPU-bound work should still go to a bounded platform-thread pool rather than a virtual-thread-per-task executor: virtual threads solve the "waiting" problem, not the "not enough cores" problem.
💻 Code example
package concurrency.iomodels; public class CooperativeSchedulingDemo { public static void main(String[] args) throws Exception { // Thread.ofVirtual().start(...) creates and immediately schedules a // virtual thread (JEP 444's builder API). Thread t1 = Thread.ofVirtual().start(() -> { System.out.println("t1 running..."); try { Thread.sleep(100); } catch (InterruptedException e) {} // Blocking sleep on a virtual thread triggers an "unmount": the // JVM detaches this thread's continuation from its carrier and // parks it, freeing the carrier to run t2 during the 100ms wait. System.out.println("t1 resumed."); }); Thread t2 = Thread.ofVirtual().start(() -> { // Because t1 yielded its carrier during sleep, t2 can run // concurrently on the freed-up carrier without waiting for t1. System.out.println("t2 running concurrently..."); }); t1.join(); t2.join(); // Expected interleaving: "t1 running...", then "t2 running // concurrently..." prints WHILE t1 is still asleep, then "t1 resumed." } }
▲ Common mistake
Running blocking calls inside an NIO event loop. A single synchronous database call or blocking file read inside a selector-based handler stalls the one thread also responsible for monitoring every other socket registered with it - the entire server stops making progress until that call returns. Every operation inside the loop has to be genuinely non-blocking, with no exceptions.
▲ Common mistake
Expecting virtual threads to speed up CPU-bound work. Virtual threads only yield at blocking calls; a tight compute loop never yields, so running it on a virtual thread provides no benefit over a platform thread - it's still bound by however many CPU cores are actually available. Keep CPU-bound work on a bounded platform-thread pool sized to the core count.
▲ Common mistake
Forgetting that virtual threads can still get "pinned." Certain operations - notably a synchronized block around a blocking call, prior to a JDK 24 fix, or a native call through the foreign function interface - prevent a virtual thread's carrier from being released during the blocking wait, silently degrading back toward the thread-per-request cost model for that code path. -Djdk.tracePinnedThreads=full surfaces exactly where this happens.
◆ Under the hood
Is reactive programming dead now that virtual threads exist? Not entirely. What virtual threads genuinely obsolete is the scalability argument for reactive - previously, handling 10,000-plus concurrent connections without exhausting OS threads required a reactive framework; now blocking code on virtual threads handles that too, and Spring MVC with virtual threads can match WebFlux's throughput for typical I/O-bound workloads. What reactive still offers that virtual threads don't provide automatically: built-in backpressure, where a subscriber explicitly signals how much demand it can handle; elegant operator composition for stream transformations; and R2DBC, a truly async database driver with no blocking and no carrier pinning at all. For most everyday web APIs, Spring Boot with virtual threads is now the simpler default - debuggable stack traces, familiar sequential code. Reactive still earns its complexity for genuine streaming workloads with backpressure requirements, or for teams already deeply invested in it; existing reactive code isn't worth migrating away from just because virtual threads exist.
Blocking I/O (thread-per-request) : A thread is parked by the OS during every I/O wait, holding roughly a megabyte of stack the whole time. Simple, debuggable code. Scales to a few hundred concurrent requests before the thread pool size becomes the throughput ceiling. Traditional Tomcat.
Thread-per-core with async NIO : A small number of Selector threads monitor thousands of channels via epoll/kqueue, touching a channel only when it's actually ready. Scales to huge connection counts with minimal memory, at the cost of callback- or reactive-style code. Netty, WebFlux, Vert.x.
Virtual threads : Ordinary blocking code; the JVM unmounts a virtual thread on a blocking call and lets its carrier OS thread run something else. Millions of concurrent logical threads on a handful of real OS threads. Spring Boot 3.2+.
Why don't virtual threads help CPU-bound work? : Virtual threads are scheduled cooperatively - they only yield their carrier at a blocking call. A tight compute loop never yields, so it behaves exactly like a platform thread and is still limited by the actual core count.
Is reactive programming obsolete? : Not entirely. Virtual threads remove the scalability argument for reactive on typical I/O-bound APIs, but reactive still offers built-in backpressure, operator composition, and truly async drivers like R2DBC that matter for genuine streaming workloads.
Want a visual for this concept?
Generate a diagram tailored to “I/O Threading Models: Blocking, Non-Blocking & Virtual” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.
Sign in to generate a visual →