beginner~4h

Images, Layers, Caching & Container Lifecycle

A Docker image is a read-only, layered filesystem template plus metadata (entrypoint, env vars, exposed ports, etc.). A container is a running (or stopped) instance of an image: the image's layers mou

Learning objectives

Explain what an image layer actually is, and why layers are shared and reused across images.
Explain the copy-on-write relationship between an image's read-only layers and a container's writable layer.
Trace a container through its full lifecycle (created, running, paused, stopped, removed) and explain what persists at each stage.

📖 Story

Imagine building a picture using a stack of transparent sheets, laid one on top of another — the first sheet draws the background, the next adds a building, the next adds a tree, and so on. Looked at from above, you see one complete picture, but underneath, it's really a stack of separate, individually-drawn sheets. A Docker image is built exactly this way: each instruction in a Dockerfile (FROM, RUN, COPY, ...) produces one layer — a sheet — and the image as a whole is just an ordered stack of these layers.

Why layers, specifically

Layers being separate, individually-identified pieces (each with its own content hash) is what makes two things possible: sharing — if two images both start FROM ubuntu:22.04, they literally share that same base layer on disk, rather than each having their own private copy — and caching — if you rebuild an image and only the last instruction changed, Docker can reuse every earlier, unchanged layer instead of rebuilding them from scratch.

A container is one more sheet, but it's the only one you can draw on

When you run a container from an image, Docker doesn't copy or modify any of the image's layers — instead, it adds one new, thin, writable layer on top, and that's the only layer the running container can actually write to. All the layers below it stay exactly as they were, read-only, shared with every other container running from that same image.

Copy-on-write, precisely

When a container's process tries to modify a file that only exists in a read-only layer below, Docker doesn't modify that shared file directly — it first copies the file up into the container's own writable layer, and the modification happens there. This is copy-on-write: reads can be satisfied directly from the shared, read-only layers underneath, but the very first write to any given file triggers a copy into the container's private layer, from which point that container sees its own private version, and every other container still sees the original, unmodified one.

The union filesystem

Docker uses a union filesystem driver (commonly overlay2 on modern Linux systems) to make a stack of separate layer directories on disk appear, to any process inside the container, as one single, seamless filesystem. Each layer is stored as its own directory tree on the host; overlay2 presents a merged view where files from higher (more recently added) layers take precedence over files with the same path in lower layers.

The "copy-up" operation

When a container process opens a file for writing that only exists in a lower, read-only layer, overlay2 performs a copy-up: the entire file (not just the changed bytes) is copied from the read-only layer into the container's writable layer, and the write proceeds against that copy from then on. This is precisely why modifying a very large file that lives in a lower layer can be surprisingly slow the first time — the whole file has to be copied up before the write can even begin, regardless of how small the actual change is.

Image layer content-addressing

Each layer is identified by a cryptographic hash of its actual content — this is what enables sharing: if two completely unrelated images happen to produce byte-for-byte identical layer content (a very common case for shared base-image layers like ubuntu:22.04), Docker recognizes the hash match and stores that layer only once on disk, referenced by both images. This also underlies the build cache: Docker hashes each Dockerfile instruction's inputs, and if that hash matches a previous build's result, it reuses the cached layer instead of re-executing the instruction.

Created — docker create (or the first phase of docker run) allocates the container's writable layer and its configuration, but doesn't start any process inside it yet.
Running — the container's main process starts inside its isolated namespaces; any file writes now trigger copy-up into the writable layer as needed.
Paused — docker pause freezes all of the container's processes using a cgroup freezer, without stopping them; memory state is preserved exactly, and processes resume from precisely where they left off on docker unpause.
Stopped — the main process exits (or is sent a stop signal), but the writable layer is NOT deleted — it stays on disk, which is exactly why docker start on a stopped container can resume with its previous filesystem state intact, and why docker logs/docker cp still work on a stopped container.
Removed — docker rm is the step that actually deletes the writable layer and frees its disk space; this is irreversible, and it's the step people most often forget to run, leading to accumulated disk usage from long-stopped containers.

Ten different application images all built FROM node:20 — that base layer stack is stored once on the host and shared across all ten images' containers, which is a large part of why running many containers from related images is far cheaper, disk-wise, than ten fully independent VMs would be.
A CI pipeline rebuilding an image after a one-line source code change — if the Dockerfile is ordered so dependency installation happens before the source COPY, the dependency-install layer is cache-hit and skipped entirely, and only the final layers rebuild, often cutting build time from minutes to seconds.
A container writing to a large log file that was baked into the base image (a genuine anti-pattern, but instructive) — the very first write to that file triggers a full copy-up of it into the writable layer, which can be a surprising, one-time latency spike on an otherwise fast container.
docker ps -a showing dozens of stopped, unremoved containers on a build server — a classic sign that cleanup (docker rm or a scheduled docker system prune) isn't part of the CI pipeline's routine, quietly consuming disk space.
docker pause/docker unpause used to briefly free up CPU for another process without losing a long-running container's in-memory state — a genuinely useful, if less common, operational tool that only makes sense once you understand the writable-layer/process-state relationship.

Order Dockerfile instructions from least-frequently-changing to most-frequently-changing — dependency installation before source code copying, for instance — so the build cache is invalidated as late as possible in the instruction sequence.
Prefer common, widely-shared base images (official node, python, postgres images) over building everything from scratch or an unusual base, to benefit from layer sharing across your own images and the broader ecosystem's cached layers.
Avoid writing large files into an image that a running container is expected to modify — if a file needs frequent writes, it likely belongs on a mounted volume (covered in the Storage chapter), not baked into a layer that would trigger an expensive copy-up.
Run docker rm (or docker run --rm for genuinely disposable containers) as a routine part of any script or CI job that creates containers, rather than letting stopped containers accumulate.
Periodically run docker system df to see actual disk usage broken down by images, containers, and volumes, and clean up what's genuinely unused with docker system prune.

⚠️ Why this keeps happening

Dockerfiles are usually written in the same order a human would naturally describe the build steps — set up dependencies, then copy the code — without necessarily realizing that the ORDER of those steps directly determines how much gets cache-invalidated on every single code change, which is why this specific mistake is both extremely common and highly fixable once someone points it out.

Copying source code before installing dependencies. Every single code change (even a one-line comment edit) invalidates the cache for the dependency-install layer too, forcing a full reinstall on every build, when the actual dependency list rarely changes.
Never running docker rm on stopped containers, letting them accumulate indefinitely on long-lived build servers, quietly consuming disk space that only becomes an emergency once the disk actually fills up.
Baking large, frequently-written files into an image layer instead of using a mounted volume — causing a surprising one-time copy-up latency spike, and unnecessarily bloating the image's read-only layers with data that was always meant to change.
Assuming docker pause stops billing/resource usage entirely — a paused container's memory is still resident and counted, it's only CPU scheduling that's frozen; it's not equivalent to stopping the container.
Not realizing shared base-image layers mean a docker rmi on one image doesn't necessarily free as much disk space as expected, if other images still reference the same shared layers underneath — the layer is only actually deleted once nothing references it anymore.

Order instructions by change frequency (rarely-changing first) — this is, by a wide margin, the single biggest lever for faster repeated builds.
Combine related RUN instructions with && where it makes sense, to avoid creating an excessive number of small layers, while still keeping logically distinct steps separate enough to benefit from partial cache hits.
Use .dockerignore to exclude files that shouldn't be part of the build context at all (like .git, local node_modules, or build artifacts) — a smaller build context means faster context-upload time to the daemon, even before any layer caching comes into play.
Use BuildKit's cache-mount feature (covered further in the Performance Optimization chapter) for package manager caches specifically, so dependency downloads persist across builds even when the dependency-install layer itself isn't cache-hit.
Measure actual image size and layer count with docker history on any image you're optimizing — it directly shows which instructions contributed the most bytes, guiding where to focus effort.

Enforce Dockerfile instruction ordering (dependencies before source) as a code-review standard, not just a personal habit — it has an outsized, compounding effect on CI time across an entire team.
Schedule regular docker system prune (with appropriate flags to avoid removing images still in active use) on build servers and hosts running many short-lived containers, rather than relying on manual cleanup.
Monitor image size trends over time in CI — a slowly growing base image or an accidentally-included large file in a layer is easy to miss build-by-build, but adds up.
Use a registry that supports layer deduplication and efficient storage, and confirm your CI's build cache (whether local or a remote cache backend) is actually being hit as expected — a misconfigured cache silently rebuilding everything from scratch is a common, invisible cost.
Document any deliberately large or frequently-written files baked into an image (if genuinely unavoidable) so a future engineer understands the copy-up cost trade-off that was accepted.

Build the same Dockerfile twice with no changes, and time the second build — confirm it's dramatically faster due to full cache reuse.
Change only a source file (not a dependency file) in a Dockerfile that copies source before installing dependencies, rebuild, and observe the dependency-install step re-running unnecessarily — then reorder the instructions and confirm it no longer does.
Run docker history on an image and identify which single instruction contributed the most bytes to the final image size.
Start a container, stop it without removing it, and run docker start on it again — confirm any files it had written earlier are still present, demonstrating the writable layer survived the stop.

✓ Quick recap

An image is an ordered stack of read-only layers, each produced by one Dockerfile instruction, identified by a content hash that enables sharing and build caching.
A running container adds exactly one thin writable layer on top of the image's read-only layers — copy-on-write means the first write to a file triggers a full copy-up into that writable layer.
Container lifecycle: created → running → (optionally paused) → stopped → removed — the writable layer persists through stop, and is only actually deleted on docker rm.
Order Dockerfile instructions from least- to most-frequently-changing to maximize build cache reuse — this is the single biggest lever for fast repeated builds.
Stopped containers still consume disk space until explicitly removed — routine cleanup (docker rm, docker system prune) is a real operational necessity, not an optional nicety.

Want a visual for this concept?

Generate a diagram tailored to “Images, Layers, Caching & Container Lifecycle” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.

Practice quiz

Next Step

Continue to Dockerfile Instructions & Multi-Stage Builds →← Back to all Docker chapters