What is a 'game day' or chaos engineering exercise, and what production gap does it catch that monitoring alone doesn't?

A game day is a deliberate, planned exercise where the team intentionally injects a failure (kills a replica, simulates a database failover, blocks a dependency) in a controlled way and observes whether the system's HA mechanisms, alerts, and runbooks actually work as designed. Monitoring tells you what is currently happening; it doesn't tell you whether your failover logic, your runbook's steps, or your team's response process actually work, because those are rarely exercised by real incidents often enough to build confidence. Game days convert untested assumptions about resilience into verif