Redis — Pub/Sub, Streams, Transactions & Lua Scripting

A Redis Stream is an append-only log of messages, each with an auto-generated, time-ordered ID — XADD stream * field value appends (the * asks Redis to generate the ID), XRANGE/XREAD read entries back. Unlike Pub/Sub, entries genuinely persist in the stream (subject to normal persistence settings, Chapter 02) until explicitly trimmed — a consumer that was offline for a deploy, a crash, or a maintenance window can reconnect and read everything it missed, because the messages never disappeared just because nobody was listening at the moment they arrived.

Consumer groups add coordination on top of that log for the common case of multiple worker processes sharing the load: XGROUP CREATE stream group $ creates a group starting from new messages only (or 0 to start from the beginning of the stream). Each worker calls XREADGROUP GROUP group consumer-name ... to receive the next unclaimed message — Redis guarantees each message in the stream is delivered to exactly one consumer within the group, so multiple workers can read from the same stream with no duplicate processing between them. Redis also tracks a pending-entries list per consumer group, and a worker must explicitly XACK a message once it's finished processing it. If a worker receives a message and then crashes before acking it, the message sits in the pending list, unacknowledged — another consumer can then XCLAIM it after a timeout and reprocess it, which is exactly how a crashed worker's in-flight message gets picked up rather than lost.

	Pub/Sub	Streams
Message history	None — miss the instant, miss it forever	Persisted in the stream until trimmed
Delivery guarantee	At-most-once, only to currently-connected subscribers	At-least-once per consumer group, with explicit ack/reclaim
Multiple workers sharing load	Every subscriber gets every message (broadcast)	Consumer groups split messages across workers, one per message
Best for	Ephemeral, best-effort notifications	Job queues, event logs, anything a missed message would actually cost you

XADD orders * order_id 5001 status "placed"

XGROUP CREATE orders order-workers $

-- each worker process:
XREADGROUP GROUP order-workers worker-1 COUNT 1 STREAMS orders >
-- ... process the order ...
XACK orders order-workers 1699999999999-0

-- recover a message a crashed worker never acked:
XCLAIM orders order-workers worker-2 30000 1699999999999-0

MULTI starts queuing commands instead of running them immediately; EXEC runs every queued command back-to-back with no other client's commands interleaved between them, guaranteed by the same single-threaded execution model Chapter 01 §1.5 covers. That's the isolation MULTI/EXEC gives you — but it's important to be precise about what it doesn't give you.

WATCH key before MULTI is what makes this genuinely useful for concurrency, not just batching: it tells Redis to monitor that key, and if any other client modifies it before this client calls EXEC, the entire transaction is aborted — EXEC returns nil, and none of the queued commands run at all. This is optimistic concurrency control, the same core idea as JPA's @Version column (Transaction Mastery, Locking/MVCC chapter) — proceed without locking, but detect at the end whether someone else changed the data first — except Redis checks the key's actual value/existence directly rather than a dedicated version field, and on a detected conflict, the caller is expected to retry the whole read-decide-write sequence from scratch rather than receiving an exception to catch.

WATCH inventory:sku-9
stock = GET inventory:sku-9   -- read outside the transaction

-- decide, in application code, whether stock is sufficient

MULTI
DECRBY inventory:sku-9 1
EXEC
-- nil → someone else modified inventory:sku-9 since WATCH; retry the whole thing
-- [result array] → succeeded, nothing else touched the key in between

EVAL script numkeys key [key ...] arg [arg ...] runs a Lua script server-side, and because of the single-threaded execution model (Chapter 01 §1.5), the entire script runs to completion as one atomic unit — no other client's command can interleave partway through, so the script can safely GET, branch in Lua, and conditionally SET/DEL in a way no sequence of plain Redis commands can replicate atomically.

The canonical real example is the lock-release problem from Chapter 04 §4.4: releasing a lock safely means checking that the token stored at the lock key still matches the token this process set, and only then deleting it — a plain GET followed by a plain DEL has a race (the lock could expire and be re-acquired by someone else in between the two calls), but a Lua script performs the check-and-delete as a single atomic step, closing that race entirely.

-- unlock.lua: only delete the lock if this process still owns it
if redis.call('GET', KEYS[1]) == ARGV[1] then
    return redis.call('DEL', KEYS[1])
else
    return 0
end

-- invoked as:
EVAL "if redis.call('GET', KEYS[1]) == ARGV[1] then return redis.call('DEL', KEYS[1]) else return 0 end" 1 lock:product:101 "req-8f3a"

	WATCH + MULTI/EXEC	Lua script (EVAL)
Where the logic lives	Application code, in your language of choice	A separate Lua script you write, test, and maintain
On conflict	Aborts (nil); caller retries the whole sequence	No conflict possible — the whole read-decide-write happens as one atomic step
Best for	Low-contention operations where a retry loop is cheap and logic is easier to keep in application code	High-contention, hot-path operations (the stampede lock, an inventory decrement) where a true atomic check-then-act is worth the extra script
Failure mode to watch for	Retry storms if contention is actually high	A slow or unbounded script blocking every other client (§5.4)

In practice, WATCH/MULTI is the default choice — it keeps business logic in application code where it's easiest to read, test, and change — and reaching for a Lua script is a deliberate step up, taken specifically when conflicts are frequent enough that constant retries would themselves become the bottleneck, or when the operation genuinely can't be expressed as "queue some commands blind" at all, the way a conditional check-then-act can't.