When designing a full producer-to-consumer pipeline (produce, persist, retry, DLT, monitor), what's the most common architectural mistake teams make early on?

Under-provisioning partition count upfront — since increasing partitions later changes the key-to-partition hash mapping, breaking the ordering guarantee for any key whose messages were relying on landing in the same partition, so partition count is one of the few Kafka decisions that's genuinely expensive to change after the fact.