Why does a 'read-process-write' consumer pattern need Kafka transactions to achieve exactly-once, not just idempotent producers?

Idempotent producers alone only prevent duplicate writes from producer retries — they don't atomically tie together 'I consumed this input message' with 'I produced this output message' as one unit. Kafka transactions wrap both the input offset commit and the output produce into a single atomic operation, so a crash between the two can't leave the system having produced output without ever recording that the input was consumed (or vice versa).