Temperature, Top-k & Top-p: What They Control and How to Tune Them
~20 min read
Temperature reshapes how sharply the model favors its top choice; top-k restricts sampling to a fixed number of candidates; top-p restricts sampling to the smallest set covering a probability mass. Three related but distinct dials for the same underlying randomness.
Temperature, Top-k & Top-p: What They Control and How to Tune Them is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Temperature reshapes the whole probability distribution before sampling — low = sharp/deterministic, high = flat/random
- •Top-k restricts the candidate pool to a FIXED COUNT of the most probable tokens (e.g. k=5)
- •Top-p (nucleus) restricts the candidate pool to the smallest set covering a PROBABILITY MASS (e.g. 90%)
- •Top-p is more adaptive than top-k — it automatically widens when the model is uncertain and narrows when confident
- •Rule of thumb: lower temperature for QA/chatbots, higher for brainstorming/creative writing