Kafka & Microservices

Before you write a line of Spring Boot code, you need the vocabulary and mental model Kafka is built on. Get this module wrong and every later module will feel like memorized incantations instead of engineering decisions.

You can't build intuition for replication and leader election by reading about them. This module gets a real multi-broker cluster running on your machine, and then breaks it on purpose.

"Call send() " and "annotate a method @KafkaListener " are one-liners. What actually happens between those calls and the broker is where most production Kafka bugs live. This module is the internals layer every later module leans on.

Every module from here on either builds a piece of this system or hardens a piece you already built. Get the shape of it into your head now, and every later "why are we doing this" question answers itself.

The rest of this site teaches Kafka. This module teaches the workflow used to actually build the reference application with an AI coding assistant — treated as a real engineering practice, not a footnote.

The first real code on this site. By the end of this module, HTTP requests into your Spring Boot app are landing as durable records in Kafka.

A Kafka producer with no tests is a producer you can't safely refactor. This module covers the three layers of testing used throughout the reference app.

A Kafka-backed REST API is still a REST API — clients need to discover and try it without reading your source code.

The producer's events are only useful once something reads them. This module wires up @KafkaListener and works through the offset-management decisions that actually matter in production.

This is where consumed events stop being ephemeral. Postgres, Flyway, and the NEW-vs-UPDATE logic that makes the consumer more than a dumb logger.

The producer's Embedded Kafka test (Module 07) proved a record lands in Kafka. This module proves the consumer turns that record into correct database state — the assertion that actually matters end to end.

Kafka got the data into Postgres. Now the consumer needs its own REST surface so the data is actually usable by clients — closing the loop on the architecture from Module 04.

Every setting in this module trades something for something. Understanding the trade, not just the property name, is what separates a config you copy-pasted from one you can defend in a design review.

A producer failure is your problem alone. A consumer failure can block an entire partition behind one bad record forever, if you don't design for it. This is the most consequential module on the site for production stability.

Most "mystery rebalance storms" in production trace back to one of the three settings on this page being wrong relative to how long your listener actually takes to run.

"Exactly once" is the most misunderstood phrase in Kafka. This module defines it precisely, then builds it.

A service that's running isn't the same as a service that's actually able to do its job. This module builds the health signal that Module 20's Kubernetes probes will later consume.

A short but necessary bridge module: before either service can be containerized (Module 19), it needs to run correctly outside the IDE.

The JAR from Module 18 now becomes a portable, versioned artifact that runs identically anywhere Docker does — the prerequisite for Module 20's Kubernetes deployment.

Docker runs one container. Production needs many, self-healing, scaled, configured per environment, and routable from outside the cluster — that's the job Kubernetes does.

Everything so far ran on a plaintext local cluster. This module is the one thing standing between this reference architecture and something you'd actually be allowed to run against a real production broker.

Every module built or hardened one piece. Here's the whole system, fully assembled — and where to go if you want to push past what this site covers.