Max Tokens, Stop Sequences & Frequency/Presence Penalties
~15 min read
Max tokens caps response length; stop sequences give you a hard exit point for structured output; frequency and presence penalties push the model away from repeating itself or toward exploring new ground.
Max Tokens, Stop Sequences & Frequency/Presence Penalties is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Max tokens is a hard cap on response length — too low truncates, too high wastes compute/cost
- •Stop sequences immediately halt generation on a match — essential for cleanly bounding structured output like JSON
- •Frequency penalty discourages reusing tokens that have appeared OFTEN (positive) or encourages it (negative)
- •Presence penalty pushes toward tokens that haven't appeared AT ALL yet, regardless of how often others have repeated
- •Frequency penalty targets 'how much' repetition; presence penalty targets 'has this appeared at all' — a subtle but real difference