advanced~2h

Outbox, Inbox & CDC

Closing Part 7 by solving Chapter 15 §3's specific dual-write problem — the pattern that makes every saga step in Chapter 17 actually reliable in practice, not just in theory.

◆ The problem, restated precisely

You want "save the order" and "publish OrderPlaced to Kafka" to be atomic, but they're two different systems — a database commit and a broker publish cannot be wrapped in one real transaction together.

The Outbox Pattern 's insight: don't try to make the database and Kafka atomic with each other at all. Instead, write the event as a row in a plain database table (the "outbox"), in the same local transaction as the order itself — which Chapter 02's Atomicity already guarantees is safe, since it's just one database. A separate process then reads unpublished outbox rows and relays them to Kafka afterward, completely decoupled from the original transaction.

The event never needs to be published atomically with the order — it just needs to be durably recorded atomically with it. A separate relay process handles the actual, less-critical publish step afterward.

CREATE TABLE outbox (
    id UUID PRIMARY KEY,
    aggregate_type VARCHAR(255), -- e.g. "Order"
    aggregate_id VARCHAR(255),   -- e.g. the order ID
    event_type VARCHAR(255),     -- e.g. "OrderPlaced"
    payload JSONB,
    created_at TIMESTAMP DEFAULT now(),
    published BOOLEAN DEFAULT false
);

@Transactional // ONE local transaction — both inserts commit together, or neither does
public void placeOrder(OrderRequest request) {
    Order order = orderRepository.save(toOrder(request));

    OutboxEvent event = new OutboxEvent(
        "Order", order.getId().toString(), "OrderPlaced", toJson(order));
    outboxRepository.save(event); // same transaction, same database, guaranteed atomic together
}

@Scheduled(fixedDelay = 500)
public void relayOutboxEvents() {
    List<OutboxEvent> pending = outboxRepository.findByPublishedFalse();
    for (OutboxEvent event : pending) {
        kafkaTemplate.send(event.getEventType(), event.getPayload());
        event.setPublished(true);
        outboxRepository.save(event);
    }
}

▲ Common mistake

The simple polling relay above itself has a smaller dual-write problem: "publish to Kafka" and "mark published=true" are two separate steps — a crash between them re-publishes the same event on the next poll. This is exactly why real production outbox implementations use CDC (§18.3) rather than a hand-written poller: it sidesteps this remaining gap entirely.

💻 Code example

CREATE TABLE outbox (
    id UUID PRIMARY KEY,
    aggregate_type VARCHAR(255), -- e.g. "Order"
    aggregate_id VARCHAR(255),   -- e.g. the order ID
    event_type VARCHAR(255),     -- e.g. "OrderPlaced"
    payload JSONB,
    created_at TIMESTAMP DEFAULT now(),
    published BOOLEAN DEFAULT false
);

Change Data Capture (CDC) reads a database's own transaction/write-ahead log (Chapter 05's WAL) directly, streaming every committed row change as an event — Debezium is the standard open-source CDC tool for this, commonly paired with Kafka Connect.

◆ Under the hood — why CDC is more reliable than polling

Debezium doesn't poll your outbox table with a SELECT at all — it reads the database's replication log (the same mechanism a replica uses to stay in sync), which only ever contains already-committed changes. This means Debezium can never observe or forward an outbox row that later gets rolled back, and — because it tracks its own position in the log durably — it can resume exactly where it left off after a crash, without double-publishing or missing events the way a naive polling+flag approach risks (§18.2's pitfall).

{
  "name": "order-outbox-connector",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "table.include.list": "public.outbox",
    "transforms": "outbox",
    "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter"
  }
}

💻 Code example

{
  "name": "order-outbox-connector",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "table.include.list": "public.outbox",
    "transforms": "outbox",
    "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter"
  }
}

◆ The problem

The Outbox Pattern solves reliable publishing. On the consuming side, Chapter 17 §6 already established that at-least-once delivery means a consumer can receive the same event more than once — the Inbox Pattern is the systematic, table-based version of that idempotency check.

@Transactional
public void handleOrderPlaced(OrderPlacedEvent event, String messageId) {
    if (inboxRepository.existsByMessageId(messageId)) {
        return; // already processed this exact message — safe no-op
    }
    inventoryService.reserveStock(event.items());
    inboxRepository.save(new InboxRecord(messageId)); // SAME transaction as the business logic — atomic together
}

Notice the inbox record is saved in the same local transaction as the business effect — exactly mirroring the outbox's own core insight: don't try to make two different systems atomic with each other, just make the idempotency check and the effect atomic within one local database transaction.

💻 Code example

@Transactional
public void handleOrderPlaced(OrderPlacedEvent event, String messageId) {
    if (inboxRepository.existsByMessageId(messageId)) {
        return; // already processed this exact message — safe no-op
    }
    inventoryService.reserveStock(event.items());
    inboxRepository.save(new InboxRecord(messageId)); // SAME transaction as the business logic — atomic together
}

Want a visual for this concept?

Generate a diagram tailored to “Outbox, Inbox & CDC” — the AI picks whichever visual (flowchart, comparison, sequence, etc.) best fits.

Practice quiz

Next Step

Continue to Kafka Transactions & Exactly-Once Semantics →← Back to all Transaction Mastery chapters