advanced~3h
Broker Hardware and Tuning
Size and tune brokers for real throughput: why disk I/O, not CPU, is usually the actual bottleneck, the broker configs that matter most, and the OS-level settings that let Kafka's design work as intended.
Learning objectives
- Beginner: Explain why Kafka's append-only log design makes disk I/O pattern, not raw CPU, the dominant hardware consideration.
- Beginner: State the role of num.partitions and why over-partitioning has real costs, not just benefits.
- Intermediate: Reason about the replication-factor tradeoff between durability and storage/network cost.
- Intermediate: Explain what segment.bytes and segment.ms control and how they interact with log compaction and retention.
- Advanced: Explain why Kafka deliberately relies on the OS page cache rather than managing its own in-process cache, and what that implies for available RAM sizing.
- Advanced: Justify the common recommendation to prefer XFS or ext4 with specific mount options for Kafka's data directories.
This is a Pro chapter
Sign in, then upgrade to Pro or Power to unlock this and the full Spring Ecosystem Mastery library.
Broker Hardware and Tuning