intermediate~2h

ExecutorService & Thread Pools

In real code you almost never create raw threads by hand -- you hand work to an ExecutorService and let it decide how many threads to use, when to create them, and how to shut down cleanly. This topic covers which pool type to reach for, how to configure one from scratch when the defaults are dangerous, how to size it, and how to shut it down without leaking threads or losing in-flight work.

Learning objectives

Choose the right Executors factory method for a given workload and explain why newCachedThreadPool() is risky at scale
Configure a ThreadPoolExecutor directly, understanding how corePoolSize, maximumPoolSize, and the work queue interact
Shut down an ExecutorService safely using the shutdown / awaitTermination / shutdownNow sequence
Apply Brian Goetz's thread pool sizing formula for CPU-bound and I/O-bound workloads
Use a virtual-thread-per-task executor for I/O-bound work and pick the right ScheduledExecutorService method

◆ The problem

Imagine a restaurant that hired and fired a brand-new chef for every single dish ordered — interview, onboarding, uniform, the works — then let them go the moment the dish was plated. That's what creating a raw Thread for every incoming task looks like at any real scale: each thread costs roughly a millisecond and a megabyte of stack just to come into existence, before it's done a single second of real work. At 1,000 requests a second, that's 1,000 threads created and destroyed every second — a full second's worth of pure overhead, and a gigabyte of stack memory churned through, every second, forever.

The obvious fix is the one every real restaurant already uses: hire a fixed crew of chefs once, and hand them orders as they come in. Thread creation cost drops to O(N) at startup instead of O(requests) over the program's lifetime, memory usage becomes a fixed N × 1MB instead of growing without bound, and — just as importantly — the number of chefs working at once is now something you control on purpose rather than something that happens to you under load.

That's what an ExecutorService is: a managed pool of reusable worker threads, a queue where tasks wait for a free worker, and a defined lifecycle for starting up and shutting down cleanly. This topic covers the shape of that pool — how big, how bounded, how it decides when to reject work outright — because every one of those decisions has a wrong default that looks fine in testing and fails only under real production traffic.

java.util.concurrent.Executors provides four one-line factory methods, each producing a differently-shaped pool underneath. Picking the wrong one for a given workload is both a classic interview trap and a classic production incident, because the code compiles and runs fine right up until real traffic patterns expose the mismatch.

newSingleThreadExecutor() creates a pool of exactly one worker thread backed by an unbounded queue — every submitted task runs on that same single thread, one after another, in FIFO order, giving you a serial task queue without writing any manual locking. newFixedThreadPool(n) creates exactly n worker threads for the lifetime of the pool; extra concurrent tasks queue rather than spawning a new thread, giving predictable, bounded resource usage — this is the most common choice for production workloads with a known, controlled concurrency ceiling. newCachedThreadPool() starts with zero threads and creates a brand-new one on the spot whenever no idle thread is available to take a task immediately, with no upper bound on how many threads it will create. newScheduledThreadPool(n) supports delayed and periodic execution on top of a small core pool.

Every one of these factories is, underneath, just a convenience wrapper around the same class — ThreadPoolExecutor — with different defaults baked in, which is exactly why understanding the raw constructor matters for anything beyond a quick prototype.

💻 Code example

package concurrency.executors;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

/**
 * A side-by-side survey of all four Executors.newXxx() factory methods.
 */
public class FourExecutorFactoryMethods {
    public static void main(String[] args) throws Exception {
        // 1. Single thread: every task runs on the SAME worker, FIFO order.
        ExecutorService single = Executors.newSingleThreadExecutor();
        single.submit(() -> System.out.println("Single-thread task running"));
        single.shutdown();   // orderly shutdown: stops accepting NEW tasks, doesn't block

        // 2. Fixed pool: exactly 4 workers for the pool's lifetime.
        ExecutorService fixed = Executors.newFixedThreadPool(4);
        fixed.submit(() -> System.out.println("Fixed-pool task running"));
        fixed.shutdown();

        // 3. Cached pool: 0 core threads, UNBOUNDED max -- creates a new
        // thread on the spot whenever no idle thread can take a task.
        ExecutorService cached = Executors.newCachedThreadPool();
        cached.submit(() -> System.out.println("Cached-pool task running"));
        cached.shutdown();

        // 4. Scheduled pool: supports delayed and periodic execution.
        ScheduledExecutorService scheduled = Executors.newScheduledThreadPool(2);
        scheduled.schedule(() -> System.out.println("Scheduled task running"), 100, TimeUnit.MILLISECONDS);
        scheduled.shutdown();   // safe to call before the delay elapses -- pending tasks still run
    }
}

The Executors.newFixedThreadPool/newSingleThreadExecutor factories use an unbounded queue internally, which is exactly what makes them risky in production: under sustained overload, that queue can grow without limit until the JVM runs out of memory, with no warning until it's too late. Production code typically constructs ThreadPoolExecutor directly, so every tunable is explicit rather than hidden behind a one-line factory call.

ThreadPoolExecutor's constructor exposes the real decision-making logic. corePoolSize is the minimum number of threads kept alive even when idle — created on demand, not all at startup. maximumPoolSize is the hard ceiling on total threads, but new threads beyond core are only created once the work queue is full, not merely non-empty. keepAliveTime controls how long threads beyond core survive without a task before being reaped. The workQueue is where tasks wait when all core threads are busy — an ArrayBlockingQueue (bounded) is the production-safe choice; LinkedBlockingQueue (unbounded) silently defeats maximumPoolSize, since the pool only grows past core once the queue rejects an offer, which an unbounded queue never does.

The full decision a task goes through on arrival: if the number of running threads is below corePoolSize, create a new thread. Otherwise, if the queue has room, enqueue the task. Otherwise, if threads are below maximumPoolSize, create a new thread anyway. Otherwise, reject the task according to the configured rejection policy.

💻 Code example

package concurrency.executors;

import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.Executors;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

/**
 * ThreadPoolExecutor configured directly, with every tunable explicit --
 * the production-safe alternative to the Executors.newXxx() factories.
 */
public class ThreadPoolExecutorProductionConfig {
    public static void main(String[] args) {
        ThreadPoolExecutor executor = new ThreadPoolExecutor(
            2,                                  // corePoolSize: 2 threads always kept alive
            4,                                  // maximumPoolSize: hard ceiling, reached only once the queue is full
            60, TimeUnit.SECONDS,               // keepAliveTime for threads beyond core
            new ArrayBlockingQueue<>(10),       // BOUNDED queue: prevents unbounded memory growth
            Executors.defaultThreadFactory(),
            new ThreadPoolExecutor.AbortPolicy()   // reject loudly instead of silently dropping or growing forever
        );

        for (int i = 0; i < 5; i++) {
            executor.submit(() ->
                System.out.println("Task running on: " + Thread.currentThread().getName())
            );
        }
        executor.shutdown();
    }
}

▲ Common mistake

Reaching for newCachedThreadPool() for internet-facing workloads. It creates a brand-new thread for every task whenever no idle thread is available, with maximumPoolSize set to Integer.MAX_VALUE — no ceiling at all. Under sustained high load — 10,000 requests a second with no idle threads to reuse — that's 10,000 new threads a second, and a very fast route to OutOfMemoryError. This is a real, recurring cause of production outages: a workload that "looked bursty and self-limiting" in testing turns out not to be, under real traffic.

▲ Common mistake

Pairing an unbounded LinkedBlockingQueue with a ThreadPoolExecutor that has a maximumPoolSize greater than corePoolSize. Because the pool only creates threads beyond core once the queue rejects an offer — something an unbounded queue never does — maximumPoolSize becomes entirely irrelevant, and the queue itself grows without bound instead, right up to OutOfMemoryError. Bounding the queue is what makes a rejection policy ever actually run.

▲ Common mistake

Forgetting to shut an executor down at all, or assuming shutdown() alone is enough. Executor worker threads are non-daemon by default, so the JVM will not exit while any are alive — simply finishing main() is not sufficient. And shutdown() is asynchronous: it stops accepting new tasks and returns immediately, while previously-submitted work may still be running. Code placed right after shutdown() with no awaitTermination() call can run while submitted tasks are still mid-execution.

▲ Edge case

DiscardPolicy fails silently. If a pool's rejection policy is DiscardPolicy instead of the default AbortPolicy, tasks arriving when both the pool and its queue are saturated vanish with no exception and no log line at all — a dangerous, hard-to-debug failure mode. AbortPolicy's loud, immediate RejectedExecutionException is usually the safer default precisely because it fails fast and visibly instead of quietly losing work.

💻 Code example

package concurrency.executors;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

/**
 * The full, safe shutdown sequence: submit -> shutdown() -> awaitTermination()
 * -> shutdownNow() as a fallback only if the timeout is exceeded.
 */
public class SafeExecutorShutdownSequence {
    public static void main(String[] args) throws Exception {
        ExecutorService executor = Executors.newFixedThreadPool(2);

        executor.submit(() -> {
            try { Thread.sleep(200); } catch (InterruptedException ignored) {}
            System.out.println("Task complete.");
        });

        executor.shutdown();   // stop accepting NEW tasks -- does not wait for running ones
        try {
            if (!executor.awaitTermination(800, TimeUnit.MILLISECONDS)) {
                executor.shutdownNow();   // fallback: forcibly interrupt anything still running
            }
        } catch (InterruptedException e) {
            executor.shutdownNow();
            Thread.currentThread().interrupt();
        }
        System.out.println("Executor fully shut down.");
    }
}

"How many threads should my pool have?" has a real, well-known answer, and it depends entirely on whether tasks spend their time computing or waiting. Brian Goetz's sizing formula: for CPU-bound tasks, Nthreads = NCPUs (or +1, to keep a core busy during occasional page faults) — adding more threads than cores gives no benefit, since they compete for the same cores and add context-switch overhead. For I/O-bound tasks, Nthreads = NCPUs × (1 + W/C), where W is wait time and C is compute time per task — a task spending 90ms waiting for every 10ms of computing (a 9:1 ratio) on a 4-core machine wants roughly 40 threads. Always read the core count from Runtime.getRuntime().availableProcessors() rather than hardcoding it; a pool sized for an 8-core development laptop but deployed unchanged into a 2-vCPU production container is either wildly oversized or leaves real capacity unused.

Beyond AbortPolicy and DiscardPolicy, CallerRunsPolicy makes the calling thread execute the rejected task itself instead of handing it to the pool — a natural form of back-pressure useful for HTTP servers, since it automatically slows down request ingestion exactly when the system is overloaded, without needing separate rate-limiting logic. DiscardOldestPolicy drops the oldest queued task and re-submits the new one, appropriate when only the latest data matters, such as sensor readings or stock prices.

ScheduledExecutorService distinguishes three scheduling shapes worth not confusing. schedule(task, delay, unit) runs a task exactly once, after a delay. scheduleAtFixedRate(task, initialDelay, period, unit) starts a new run every period, measured from the start of one run to the start of the next — if a single execution runs longer than period, the next run queues immediately after it finishes rather than waiting for its "scheduled" slot, silently losing the clock-aligned guarantee. scheduleWithFixedDelay(task, initialDelay, delay, unit) instead waits delay after one execution finishes before starting the next, guaranteeing a gap regardless of how long any single run took — usually the safer default when a task's duration can vary.

Every sizing formula in the previous section exists to solve one underlying problem: platform threads are expensive, so you have to carefully ration a small number of them across many I/O-bound tasks. Virtual threads (Java 21+) sidestep the problem instead of solving it — they're cheap enough that a task can simply get its own thread, with no sizing calculation needed at all. Executors.newVirtualThreadPerTaskExecutor() creates a brand-new virtual thread for every submitted task instead of reusing a bounded pool of platform threads, and the JVM manages the small number of underlying carrier threads automatically.

If you're on Spring Boot 3.2 or later, setting spring.threads.virtual.enabled=true switches the embedded servlet container to handle every incoming HTTP request on its own virtual thread — the identical pattern, applied automatically across an entire application without touching request-handling code. Don't expect a benefit for CPU-bound work, though: virtual threads still ultimately run on a limited number of carrier platform threads, so tight computational loops gain nothing from switching to them; the benefit is specifically for workloads dominated by blocking I/O.

ScheduledExecutorService is the modern replacement for the older java.util.Timer/TimerTask API, for one decisive reason: Timer runs all of its scheduled tasks on a single background thread, and if any one task throws an uncaught exception, that thread dies and every other task scheduled on that Timer silently stops firing forever. ScheduledExecutorService pools multiple worker threads, so one task's failure doesn't take the others down with it — the standard choice for cron-like or periodic background jobs: health checks, cache eviction sweeps, retry timers, heartbeats.

Since Java 19, ExecutorService implements AutoCloseable, so try (ExecutorService exec = ...) { ... } runs the shutdown-and-await sequence automatically when the block exits — a small syntactic convenience that removes an entire category of "forgot to shut the executor down" bugs.

💻 Code example

package concurrency.executors;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

/**
 * newVirtualThreadPerTaskExecutor() -- a fresh virtual thread per task,
 * no pool sizing required. try-with-resources auto-closes it, blocking
 * until every submitted task has finished.
 */
public class VirtualThreadPerTaskExecutorDemo {
    public static void main(String[] args) {
        try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
            for (int i = 0; i < 10; i++) {
                final int taskId = i;
                executor.submit(() ->
                    System.out.println("Virtual thread task " + taskId + " on: " + Thread.currentThread())
                );
            }
        }   // block exits here only after every task has completed
        System.out.println("All virtual thread tasks completed.");
    }
}

Q: Why is creating a new raw Thread for every incoming task a bad idea at scale?

A: Thread creation costs roughly a millisecond and a megabyte of stack. At 1,000 tasks per second, that's a full second of pure creation overhead and a gigabyte of stack churn every second, with no reuse. A thread pool creates a fixed number of threads once and reuses them across many tasks.

Q: Why is newCachedThreadPool() risky for internet-facing workloads?

A: It has no upper bound on threads (maximumPoolSize = Integer.MAX_VALUE) and creates a new thread whenever no idle one is available. Under sustained high load with no idle threads to reuse, it can create thousands of threads per second and exhaust memory.

Q: Why does pairing an unbounded queue with a large maximumPoolSize not actually let a pool grow?

A: ThreadPoolExecutor only creates threads beyond corePoolSize once the work queue rejects an offer -- something an unbounded queue never does. The pool silently stays at corePoolSize while the queue itself grows without bound instead, risking OutOfMemoryError under sustained overload.

Q: What is the correct, safe sequence for shutting an ExecutorService down?

A: Call shutdown() to stop accepting new tasks, then awaitTermination(timeout) to block until running tasks finish or the timeout expires, and only fall back to shutdownNow() (which forcibly interrupts running tasks) if that timeout is exceeded. Since Java 19, try-with-resources on an ExecutorService runs this sequence automatically.

Q: How do you size a thread pool, and why don't virtual threads need sizing at all?

A: CPU-bound: threads roughly equal to core count. I/O-bound: threads = cores x (1 + wait time / compute time), since most threads spend their time idle waiting. Virtual threads sidestep the calculation entirely -- they're cheap enough to give every task its own thread, with the JVM managing a small pool of carrier threads underneath automatically.

Want a visual for this concept?

Generate a diagram tailored to “ExecutorService & Thread Pools” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.

Practice quiz

Next Step

Continue to Callable, Future & CompletionService →← Back to all Java Concurrency & Multithreading chapters