Concurrency Patterns & Best Practices

The simplest possible server model: one thread handles one request from start to finish — blocking calls and all — reading like ordinary sequential code with no callbacks anywhere. It fell out of favor once platform-thread limits made it unscalable past a few hundred to a couple thousand concurrent requests, which is exactly what pushed a generation of Java services toward reactive frameworks instead. Virtual threads make this model fashionable again: same conceptual shape, one thread per request, but each request's thread is now cheap enough that the old ceiling simply doesn't apply. The JVM unmounts a blocked request's virtual thread from its carrier and remounts it on completion, so the code stays simple and blocking while the underlying throughput scales the way a reactive rewrite used to be needed for. Spring Boot 3.2+ makes this an application-wide switch via spring.threads.virtual.enabled=true, with no change to how request-handling code is written.

The pattern behind nearly every production thread pool: a fixed number of workers, a bounded queue in front of them, and a deliberate, explicit answer to "what happens when both are full?" A bounded queue matters as much as the worker count — an unbounded one just defers the problem, letting memory grow without limit under sustained overload instead of surfacing backpressure early.

CallerRunsPolicy is a particularly elegant rejection policy because it doesn't lose work or need extra buffering — once both the workers and the queue are full, the submitting thread runs the overflow task itself. That naturally throttles the producer: the thread that would otherwise be submitting more work is now busy executing the task it just tried to submit, so submission rate automatically slows to match the pool's actual processing capacity, with no separate rate-limiting mechanism required.

Break a larger unit of work into sequential stages, each running concurrently on its own thread, communicating only through queues — never through shared mutable state guarded by a lock. A BlockingQueue supplies every bit of the synchronization this pattern needs: put() blocks if the queue is full, take() blocks if it's empty, and that's the entire hand-off protocol between one stage and the next, with no explicit lock or wait/notify anywhere in sight.

A pipeline's overall throughput is set by its slowest stage, exactly like an assembly line moves at the pace of its slowest station. The usual compensation is running more worker threads on the expensive stage — an expensive "enrichment" stage might run eight threads pulling from its input queue while a cheap "validation" stage in front of it needs only two. Each queue also supplies natural backpressure automatically: a fast upstream stage can never run arbitrarily far ahead of a slow downstream one and exhaust memory, because its put() calls simply start blocking once the queue between them fills up.

Pipeline connects exactly one producer to exactly one consumer per queue. Publish-Subscribe is the pattern for the opposite shape: one event, broadcast to an arbitrary number of independent listeners, each reacting without knowing or caring that any other listener exists. A thread-safe list of subscribers — appended to rarely (subscribe/unsubscribe) and iterated constantly (publish) — is the core data structure underneath, with CopyOnWriteArrayList a natural fit for exactly that read-heavy, write-rare access pattern.

The simplest form dispatches to each subscriber directly on the publishing thread — straightforward, but a slow subscriber directly slows down the publisher, and one subscriber throwing an exception can prevent later subscribers from ever being notified unless each call is individually wrapped in its own try/catch. Asynchronous dispatch — submitting each subscriber notification as its own task to an executor — decouples publisher speed from subscriber speed entirely, at the cost of losing strict delivery-ordering guarantees.

This in-process pattern is exactly right for decoupling components within a single JVM — publishing domain events like an order being placed or a user registering to several in-process listeners handling logging, metrics, and cache invalidation. Once subscribers need to live in a different process, need guaranteed delivery across a crash, or need to replay events after the fact, that's the signal to reach for an actual message broker like Kafka or RabbitMQ instead — the in-process version trades durability and cross-process reach for near-zero latency and zero operational overhead.

Named after a ship's watertight compartments: if one section floods, the ship doesn't sink, because the flooding is contained. Applied to threads: if every downstream call shares one thread pool, and one downstream service turns slow or unresponsive, every thread in that shared pool can end up blocked waiting on it, starving completely unrelated, perfectly healthy calls of any thread to even run on. Giving each downstream dependency its own fully separate pool means a hung or slow call to one service can never consume threads reserved for another.

With virtual threads, the isolation concern usually shifts from pools to permits, since a bounded pool for the thread count itself becomes largely unnecessary. A Semaphore per downstream service plays the same isolating role a separate ExecutorService used to: acquiring a permit before calling one service, capped independently from the permit budget for a different service, so a struggling dependency can never exhaust the concurrency budget reserved for a healthy one.

Bulkhead contains a slow dependency's damage. Circuit Breaker goes a step further: it stops calling a dependency that's clearly failing at all, for a while, instead of continuing to send doomed requests that will only time out anyway and tie up threads or permits waiting on a response that isn't coming.

A circuit breaker is a three-state machine. Closed is normal operation — requests flow through as usual while the breaker counts recent failures; once failures cross a configured threshold (say, half of the last twenty calls), it trips to Open. Open rejects every call immediately, with no attempt made to reach the actual dependency at all, typically returning a fallback value or a fast error instead — this is what actually protects the calling application, since threads or permits are never spent waiting on a dependency already known to be unhealthy. After a configured cooldown, the breaker moves to Half-Open, where a small number of test requests are allowed through to check whether the dependency has recovered: if they succeed, the breaker resets to Closed; if they fail, it trips back to Open and the cooldown restarts.

Bulkhead and Circuit Breaker solve genuinely different problems and are typically layered together rather than treated as alternatives. A bulkhead limits the blast radius of a slow dependency but doesn't stop repeatedly hammering a dependency that's already known to be down. A circuit breaker stops the hammering but says nothing on its own about isolating one dependency's resource usage from another's. Production resilience code typically applies both — a bulkhead per downstream dependency, with a circuit breaker wrapping each call into it — exactly the combination libraries like Resilience4j package as separate, composable decorators around the same underlying call.

Every pattern above manages threads touching shared, changing state. Immutability sidesteps the entire problem at its root: if an object's state can never change after construction, there's nothing left for concurrent threads to race on. No lock, no volatile, no defensive copying required on read — the best synchronization is no synchronization, achieved by removing the thing synchronization exists to protect in the first place.

Four ingredients make a class genuinely immutable: every field private final, assigned exactly once inside the constructor; no setters; the class itself not meaningfully subclassable into a mutable variant; and any field that itself holds a reference type pointing to an immutable type — a mutable field like an array or a plain List would need defensive copying on the way in and out to preserve the guarantee. The final keyword here is more than documentation: the Java Memory Model specifically guarantees that final fields set inside a constructor are visible to any thread that later observes a reference to the fully-constructed object, which is what makes safe publication of an immutable object work with zero additional volatile or locking. Java's record keyword, from Java 16 onward, is the idiomatic shorthand for writing exactly this recipe with far less boilerplate.