4 LLM Text Generation Strategies + SLED
Greedy, Multinomial Sampling, Beam Search, and Contrastive Search — the 4 core strategies for picking the next token — plus SLED (Self-Logits Evolution Decoding), a bonus technique that improves factuality using every layer's predictions.
KV Parameters:
4 LLM Text Generation Strategies (+ SLED)
Each strategy trades off speed, quality, and diversity differently when turning next-token probabilities into an actual output sequence.
| Mechanism | Strength | Weakness | |
|---|---|---|---|
| Greedy | Always picks the highest-probability token at each step | Fast, deterministic, cheap | Prone to repetition loops, no diversity |
| Multinomial sampling | Randomly samples from the probability distribution | Produces diverse, less repetitive output | Can wander into low-probability, incoherent territory |
| Beam search | Explores k parallel paths, prunes low-scoring ones, keeps the best complete path | Higher-quality complete sequences than greedy | More compute (k× the paths), can still favor generic output |
| Contrastive search | Actively avoids revisiting previously-generated context | Reduces repetition without pure randomness | More complex scoring, extra compute per step |
| SLED (bonus) | Aggregates predictions from every model layer, not just the final one, toward a consensus | Improves factuality by using the full model's internal signal | Requires access to intermediate layer outputs, added complexity |
▶📚 Prerequisites(1)
🎓 Learning objectives
- •Explain why greedy decoding often produces repetitive output
- •Contrast beam search's global-sequence optimization against greedy/sampling's single-step focus
- •Explain how Contrastive Search balances fluency and diversity using a similarity penalty
- •Explain SLED's layer-wise logit consensus mechanism and why it improves factual accuracy
What is it?
Every time you prompt an LLM, it doesn't 'know' the whole sentence in advance — it predicts the next token step by step. But predicting probabilities alone isn't enough; you still need a strategy for actually picking which token to use at each step, and different strategies produce very different styles of output. There are 4 common strategies: Greedy (always pick the single highest-probability token), Multinomial Sampling (sample from the probability distribution, controlled by temperature), Beam Search (track multiple candidate sequences to approximate global sequence-level optimization), and Contrastive Search (balance fluency with diversity via a repetition-penalizing similarity check) — plus a bonus technique, SLED (Self-Logits Evolution Decoding), which improves factual accuracy by considering how predictions evolve across all of a model's layers, not just the final one. Note: greedy decoding and beam search are covered briefly elsewhere (see the companion Text Generation: Decoding & Sampling topic) — this topic gives fuller treatment specifically to Contrastive Search and SLED, both of which are covered nowhere else.
Why it exists
Predicting a probability distribution over the vocabulary at each step doesn't by itself tell you what to actually generate — you need a decision rule. The simplest rule (always take the top token) has a specific, well-documented failure mode: it leads to repetitive sentences, since the model tends to get stuck reinforcing its own most-recent output. Each successive strategy in this list exists to fix a specific limitation of the strategies before it: sampling introduces randomness to escape repetition loops; beam search exists because greedy/sampling only ever optimize the immediate next token, not the whole sequence's joint probability; contrastive search exists because even beam search and sampling can still produce repetitive or 'stuck in a loop' output in longer generations; and SLED exists because all of the above rely purely on the final layer's logits, which can lose factual signal present in earlier layers as the model goes deeper.
Problem it solves
Greedy solves nothing new (it's the naive baseline) but its repetitive-output problem motivates the others. Multinomial sampling solves greedy's repetition problem by introducing controlled randomness. Beam search solves the 'only optimizing one token at a time' problem — both greedy and sampling only focus on the most immediate token, when what actually matters is maximizing the probability of the whole generated sequence, which requires knowing future conditionals that pure step-by-step decoding can't see; beam search approximates this by keeping multiple candidate partial sequences alive simultaneously. Contrastive search solves the residual 'stuck in a loop' problem that can still affect longer generations even with sampling or beam search, by explicitly penalizing candidate tokens that are too similar to what's already been generated. SLED solves a different, factuality-specific problem: because final-layer logits alone can favor fluent-but-inaccurate outputs (since factual signal from earlier layers can fade by the final layer), SLED nudges the final prediction toward a consensus across all layers' predictions.
Intuition
Greedy decoding is like a hiker who, at every fork in the trail, always takes whichever path looks steepest downhill right now — fast, simple, but prone to walking in circles around a local valley rather than reaching the actual destination. Multinomial sampling is the same hiker occasionally taking a less-obviously-downhill path on purpose, escaping repetitive loops but risking wandering off course. Beam search is a small search party splitting up to explore several promising paths simultaneously, keeping the best few alive and abandoning the rest, so a path that starts slightly uphill but leads somewhere much better isn't missed. Contrastive search is a hiker who specifically avoids retracing ground they've already covered, actively steering away from paths too similar to where they've already been. SLED is like checking in with several trail markers positioned at different points along the route (not just the final destination sign) and nudging your decision toward where most of them agree, rather than trusting only the very last, potentially misleading, signpost.
Analogy
Think of writing an essay one word at a time. Greedy is always writing whatever single word feels most natural right now, without any lookahead — you can end up stuck rephrasing the same idea repeatedly. Multinomial sampling is occasionally choosing a less-obvious-but-still-reasonable word to keep things fresh. Beam search is like drafting several possible next sentences in parallel, keeping only the most promising few, since sometimes a slightly awkward opening word leads to a much better overall sentence. Contrastive search is like an editor specifically flagging 'you already said something very similar to this two sentences ago' and nudging you toward a genuinely different phrasing. SLED is like getting a second opinion from every earlier draft of the sentence you've been mentally composing, not just your final instinct, and going with whatever most of those drafts agree on.
Technical explanation
(1) Greedy strategy: at each step, choose the single token with the highest probability from the model's probability vector, then autoregress (feed it back in and repeat) — simple but often leads to repetitive sentences since a locally-optimal choice at each step doesn't account for how it constrains future options.
(2) Multinomial sampling strategy: instead of always picking the top token, sample from the full probability distribution, with the temperature parameter controlling how random this sampling is.
(3) Beam search: both greedy and sampling only focus on the immediate next token, but what actually matters is maximizing the probability of the whole generated sequence — computing this properly would require knowing future conditionals (what comes after each candidate token), which isn't available at decode time. Beam search approximates this global maximization by, at each step, expanding the top-k partial sequences (the 'beam') rather than just the single best one — some beams may start with a less probable token but lead to a much higher-probability completion overall; by keeping multiple alternatives alive, beam search explores more of the probability tree than either greedy or single-path sampling, and is widely used in tasks like machine translation where correctness matters more than creative variation.
(4) Contrastive search: a newer method balancing fluency with diversity — at each step, the model considers candidate tokens and applies a penalty to any candidate that is too similar to what's already been generated (measured via representation similarity, not just exact repetition), then selects the token that best balances raw probability against this diversity penalty; this prevents 'stuck in a loop' problems while keeping coherence high, and is especially effective for longer generations like stories, where repetition can easily creep in over many tokens.
Bonus — SLED (Self-Logits Evolution Decoding): all of the above strategies rely on the logits produced by the model's final layer, which is the standard way Transformers generate text. The issue is that factual signals present in earlier layers can fade as the model goes deeper, leading the final layer to favor fluent but occasionally inaccurate outputs. SLED introduces a small but meaningful change: instead of using only the final layer's logits, it looks at how logits evolve across ALL layers — each layer contributes its own prediction, SLED measures how closely these predictions agree (a layer-wise consensus), and then nudges the final logits toward this consensus before selecting the next token. This requires no retraining, no extra data, and no additional compute infrastructure — by leveraging the model's full internal knowledge rather than only its very last step, SLED produces more grounded and factual generations using the exact same underlying architecture.
Architecture
Each strategy operates as a different decision policy sitting at the same point in the generation loop — after logits are computed for the current step, before the next token is finalized. Greedy and multinomial sampling are single-path, single-step-optimal policies. Beam search maintains k parallel candidate sequences simultaneously, pruning to the top-k at every step. Contrastive search adds a similarity-based scoring term computed against the already-generated sequence, applied alongside raw token probability. SLED is architecturally distinct from the other 4 — rather than changing HOW a token is selected from a given probability distribution, it changes WHERE that distribution comes from in the first place, aggregating signal across every transformer layer rather than reading only the final layer's output; SLED can in principle be combined with any of the other 4 selection strategies once its layer-consensus-adjusted logits are produced.
Workflow
- For tasks where correctness and precision matter more than creative variation (machine translation, factual Q&A, structured extraction), consider beam search or low-temperature greedy/sampling.
- For open-ended, creative tasks (brainstorming, story generation, chat), use multinomial sampling with temperature tuned to the desired creativity level.
- If you observe repetitive or 'stuck in a loop' output in longer generations even after tuning temperature/sampling parameters, switch to contrastive search, which specifically targets this failure mode.
- If factual accuracy is a priority and you're seeing fluent-but-wrong outputs, consider layering SLED on top of whichever base decoding strategy you're using — it requires no retraining and can be added without architectural changes.
- For chat-model production use cases specifically, note that beam search is comparatively rarely used (it tends toward generic, length-biased outputs for open-ended conversation) — reserve it for tasks like translation where a single, precise, correct-ish output matters more than natural conversational variety.
Example
import torch import torch.nn.functional as F
Greedy
def greedy_step(logits: torch.Tensor) -> int: return torch.argmax(logits).item()
Multinomial sampling
def sample_step(logits: torch.Tensor, temperature: float = 0.8) -> int: probs = F.softmax(logits / temperature, dim=-1) return torch.multinomial(probs, 1).item()
Simplified contrastive search: penalize similarity to recent generations
def contrastive_step(logits: torch.Tensor, candidate_embeddings, generated_embeddings, alpha=0.6) -> int: probs = F.softmax(logits, dim=-1) max_similarity = torch.tensor([ cosine_similarity(c, generated_embeddings).max() for c in candidate_embeddings ]) score = (1 - alpha) * probs - alpha * max_similarity # balance prob vs. diversity return torch.argmax(score).item()
Conceptual SLED: nudge final logits toward cross-layer consensus
def sled_adjust(all_layer_logits: list[torch.Tensor]) -> torch.Tensor: final_layer = all_layer_logits[-1] consensus = torch.stack(all_layer_logits).mean(dim=0) # simplified consensus return final_layer + 0.3 * (consensus - final_layer) # nudge toward consensus
Real-world usage
Machine translation systems (Google Translate, professional translation tools) commonly use beam search specifically because a single, precise, high-probability-sequence translation is preferred over creative variation. Chat products (ChatGPT, Claude's consumer interfaces) predominantly use multinomial sampling with tuned temperature/top-p rather than beam search or pure greedy, since natural conversational variety matters more than finding the single globally-optimal response sequence. Long-form story-generation and creative-writing tools have adopted contrastive search specifically to combat the well-documented 'repetition creep' problem that plagues naive sampling over very long outputs. SLED-style factuality-focused decoding techniques are increasingly explored by teams building fact-sensitive applications (medical/legal Q&A assistants, citation-grounded research tools) where the standard fluency-optimized final-layer decoding measurably increases hallucination risk.
Trade-offs
Greedy is fastest and cheapest but the most prone to repetitive, low-quality output for anything beyond very short generations. Multinomial sampling adds controllable randomness at essentially no extra compute cost, but pure sampling with no other safeguards can occasionally wander into low-quality territory. Beam search meaningfully improves sequence-level quality for precision-sensitive tasks but costs k times the compute of greedy (maintaining k candidate beams) and tends to produce more generic, length-biased output for open-ended tasks, which is why it's rarely used for chat. Contrastive search adds a similarity-computation overhead at each step compared to plain sampling, in exchange for meaningfully reducing repetition in long generations. SLED adds essentially zero additional compute or retraining cost and can be layered onto any other strategy, but requires access to intermediate layer activations, which not all inference serving setups expose by default.
Visual explanation
A comparison diagram with 4 (+1 bonus) branches from a shared starting point. Greedy: a single straight arrow always following the highest-probability branch at each step — with a warning icon showing it looping back on a repeated phrase. Multinomial sampling: the same tree but the path occasionally branches onto a lower-probability option, shown as a slightly wobbly line. Beam search: multiple parallel paths (a 'beam' of width k) explored simultaneously, with lower-scoring paths pruned at each step and the best-scoring complete path highlighted at the end. Contrastive search: a path that actively avoids looping back near previously-visited nodes, shown with a repulsion effect pushing the path away from earlier points. SLED (bonus): a vertical cross-section showing predictions from every model layer (not just the final one) converging toward a 'consensus' direction that the final output is nudged toward.
Advantages
- —
Provides 4 distinct decision policies (plus SLED) covering the full spectrum from deterministic-and-precise to creative-and-diverse generation needs
- —
Beam search's global sequence approximation meaningfully outperforms single-step-optimal strategies for precision-critical tasks like translation
- —
Contrastive search directly targets the specific, well-documented 'stuck in a loop' repetition failure mode of long-form generation
- —
SLED improves factual grounding with zero retraining, zero extra data, and no architectural changes, layering onto any existing decoding strategy
Disadvantages
- —
Greedy decoding's repetition problem makes it unsuitable for most open-ended generation tasks beyond very short outputs
- —
Beam search costs k times the compute of greedy/sampling and tends toward generic, length-biased outputs unsuited to open-ended chat
- —
Contrastive search's similarity computation adds per-step overhead compared to plain sampling
- —
SLED requires access to intermediate layer activations, which not every inference serving stack exposes by default
Common mistakes
- —
Using pure greedy decoding for open-ended or creative generation tasks and being surprised by repetitive, low-quality output
- —
Applying beam search to conversational chat applications, producing generic, less natural-sounding responses compared to well-tuned sampling
- —
Not reaching for contrastive search when observing 'stuck in a loop' repetition in long-form generations, instead only adjusting temperature which doesn't directly address this failure mode
- —
Assuming the final layer's logits are the complete picture of what the model 'knows,' missing SLED's insight that earlier-layer factual signal can be lost by the final layer
- —
Treating decoding strategy choice as a one-time default rather than matching it to the specific task's precision-vs-creativity requirements
📂 Subtopics
Greedy Decoding: Always Pick the Highest-Probability Token, and Why It Fails
The simplest possible decoding strategy — always take the single most likely next token. Fast and deterministic, but it produces repetitive, low-quality text because it never considers the whole sequence, only the immediate next step.
~10 min
Beam Search: Keeping Top-k Sequences Alive, and Its Trade-offs vs. Greedy
Beam search approximates true whole-sequence maximization by keeping the top-k partial sequences alive at every step, rather than committing to one path like greedy decoding — at the cost of k times the compute.
~15 min
Sampling Methods in Practice: Temperature Scaling with Top-p/Top-k
Multinomial sampling picks from the probability distribution instead of always taking the top token — and in practice, it's almost never used alone. Temperature, top-p, and top-k are the three dials that shape the distribution being sampled from.
~15 min
Contrastive Search & SLED: Newer Strategies and When to Use Them
Contrastive search penalizes tokens too similar to what's already been generated to fight repetition while keeping coherence. SLED goes further, using EVERY transformer layer's predictions — not just the final one — for more factually grounded generation.
~15 min