beginner~4h

Images, Layers, Caching & Container Lifecycle

A Docker image is a read-only, layered filesystem template plus metadata (entrypoint, env vars, exposed ports, etc.). A container is a running (or stopped) instance of an image: the image's layers mou

Images are built layer-by-layer, where each Dockerfile instruction that changes the filesystem (FROM, RUN, COPY, ADD) produces a new immutable layer, identified by a content hash. Layers are content-addressable and shared across images — if two images both start FROM the same base, that base layer is stored once on disk and reused.

The container lifecycle is the state machine a container moves through: created -> running -> paused/unpaused -> stopped -> removed. Understanding this state machine is essential for writing correct restart policies, healthchecks, and orchestration logic.

Each Dockerfile instruction that touches the filesystem is executed in a temporary container, and the resulting filesystem diff is committed as a new layer with its own SHA256 digest.
Layers are stacked using a union filesystem (overlay2 on modern Linux): lower layers are read-only, and the topmost container layer is a writable 'diff' directory that captures any runtime changes.
When a container writes to a file that exists in a lower (read-only) layer, the storage driver performs copy-up: it copies the file into the writable layer first, then applies the write — this is why heavy writes to pre-existing large files are slow in containers.
The Docker build cache works layer-by-layer: if an instruction and its inputs (build context files for COPY/ADD) are unchanged since the last build, Docker reuses the cached layer instead of re-executing it — and skips every instruction after the first cache miss.
docker commit and docker save/docker load operate on this same layered model: an image is really just a manifest (JSON) referencing an ordered list of layer digests plus a config blob.
Container lifecycle transitions are driven by the init process (PID 1) inside the container: when PID 1 exits, the container moves to Exited, regardless of any background child processes still notionally running — this is why improper signal handling in PID 1 causes 'zombie' or hung containers.

docker build reads the Dockerfile top to bottom, one instruction at a time.
For each instruction, Docker checks the build cache: same base image digest + same instruction text + same input files (for COPY/ADD) => cache hit, reuse existing layer.
On a cache miss, Docker spins up a temporary container from the previous layer, executes the instruction inside it, snapshots the filesystem diff as a new layer, and discards the temporary container.
After the final instruction, Docker assembles the manifest: an ordered list of layer digests plus image config (entrypoint, cmd, env, exposed ports, labels).
docker run takes this manifest, mounts all layers read-only via the storage driver, adds a new empty writable layer, and starts the configured process as PID 1 inside fresh namespaces.
While running, any file writes by the container go into the writable layer only — the underlying image is never modified, which is why the same image can safely back hundreds of independent containers.
On docker stop, Docker sends SIGTERM to PID 1, waits a grace period (default 10s), then sends SIGKILL if the process hasn't exited.
On docker rm, the writable layer is deleted permanently — any un-persisted data in that layer is lost unless it lived in a volume or bind mount.

Base image standardization: a platform team publishes a hardened company/java21-base image; every service's Dockerfile starts FROM it, so a single security patch rebuild propagates everywhere.
Build cache exploitation in CI: ordering COPY pom.xml / package.json before COPY of full source code so dependency-resolution layers are cached and only re-run when dependencies actually change.
Debugging a 'container exits immediately' issue by understanding that PID 1 exiting (e.g., a script that runs and finishes) ends the container even if you expected a long-running process.
Disk-usage audits using docker system df -v to find which images share base layers versus which are needlessly duplicating gigabytes of layers.

Order Dockerfile instructions from least-frequently-changed to most-frequently-changed so cache hits are maximized (dependencies before source code, source code before final small config tweaks).
Combine related RUN commands with && to avoid creating many small layers that each carry filesystem overhead (e.g., apt-get update && apt-get install -y X && rm -rf /var/lib/apt/lists/* in one RUN).
Use .dockerignore aggressively — every file in the build context is hashed for cache-key purposes, and unnecessary files (node_modules, .git, target/) bloat both context upload time and cache invalidation.
Use multi-stage builds (Chapter 3) to discard build-time layers entirely from the final image rather than trying to 'clean up' within the same stage.

Believing docker rm or docker stop deletes the image — it only affects the container; the image and its layers remain on disk until docker rmi.
Putting COPY . . before dependency installation, which invalidates the dependency-install cache layer on every single source code change, even a one-line comment edit.
Assuming data written inside a running container persists across docker rm — without a volume or bind mount, it is gone the moment the writable layer is deleted.
Confusing docker stop (graceful, SIGTERM then SIGKILL) with docker kill (immediate SIGKILL) and being surprised by data corruption from killing a database container without a clean shutdown.

Minimize layer count and layer size — fewer, smaller layers pull and push faster, and overlay2 has less work to do mounting them.
Use BuildKit (default in Docker 23+) which parallelizes independent build stages and supports more efficient cache mounts (--mount=type=cache) for package manager caches (npm, pip, maven).
Avoid writing large amounts of data into the container's writable layer at runtime for write-heavy workloads — copy-up semantics on overlay2 make this slower than a dedicated volume.

Tag images immutably by digest or a unique build identifier (git SHA, build number) in production deployments — never deploy by floating tags like latest or stable.
Set explicit restart policies (--restart unless-stopped or on-failure:5) rather than relying on manual intervention when a container crashes.
Periodically run docker system df and scheduled docker system prune (with care, scoped by label/age) on long-lived hosts to prevent disk exhaustion from accumulated unused layers.

Build the same Dockerfile twice with no changes, then once with a one-line source change. Use docker history to see how many layers were reused vs rebuilt.
Start a container, modify a file inside it, then run docker diff <container> to see exactly what changed in the writable layer.
Run a container with a script that exits after 5 seconds as the entrypoint; observe with docker ps -a how quickly it moves to Exited, confirming PID-1-driven lifecycle.
Use docker system df -v before and after pruning to quantify reclaimed space from dangling layers.

Image = stack of immutable, content-addressed layers + metadata. Container = image layers (read-only) + one writable layer + isolated process.
Build cache reuses layers when instruction + inputs are unchanged; one cache miss invalidates everything after it.
Order Dockerfiles from stable to volatile instructions to maximize cache hits.
Container lifecycle is driven by PID 1: created -> running -> paused/stopped -> removed.
Deleting a container destroys its writable layer permanently — persistent data needs volumes, not the container filesystem.

Want a visual for this concept?

Generate a diagram tailored to “Images, Layers, Caching & Container Lifecycle” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.

Practice quiz

Next Step

Continue to Dockerfile Instructions & Multi-Stage Builds →← Back to all Docker chapters