advancedTop 15 System Design Questions
Design a logging and analytics system for 1TB of logs per day.
Ingestion: Kafka as durable buffer; handles burst traffic Storage: Cassandra for time-series raw logs (high write, TTL for retention) Aggregation: Spark Streaming on Kafka → pre-aggregate to Cassandra summary tables Search: Elasticsearch for full-text log search (last 7 days) Redis: real-time counters (errors per minute), alerting thresholds MongoDB: stored queries, dashboard configs, alert rules
This is a Pro chapter
Sign in, then upgrade to Pro or Power to unlock this and the full Databases Mastery library.
Design a logging and analytics system for 1TB of logs per day.