During a postmortem, the team discovers their nightly database backup script has been silently failing for three weeks due to a credential rotation that broke the backup job's auth, and nobody noticed

Diagnose why this went undetected and propose changes to prevent recurrence. Expected reasoning: the core gap is that backup success was never itself monitored/alerted on — the job's failure was invisible because nothing checked whether it actually completed. Recommend: alerting specifically on backup-job failure (not just on the cron running, but on a verified successful backup artifact existing with the expected size/recency), periodic automated restore-verification drills that would have caught the silent breakage by attempting to actually use a recent backup, and treating the backup pipeli