Greedy Decoding: Always Pick the Highest-Probability Token, and Why It Fails
~10 min read
The simplest possible decoding strategy — always take the single most likely next token. Fast and deterministic, but it produces repetitive, low-quality text because it never considers the whole sequence, only the immediate next step.
Greedy Decoding: Always Pick the Highest-Probability Token, and Why It Fails is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Greedy decoding always picks the single highest-probability token at each step — deterministic and fast
- •It only optimizes the immediate next token, never the quality of the whole sequence
- •This myopic, no-lookahead behavior is exactly why it's prone to repetitive sentences and loops
- •There's zero randomness and zero recovery mechanism once the generation starts down a repetitive path
- •Best suited to latency-critical or narrow-answer tasks; rarely the right default for open-ended generation