Max Tokens, Stop Sequences & Frequency/Presence Penalties

Beyond the randomness-controlling trio (temperature/top-k/top-p), four more parameters shape a generation in different ways.

Max tokens is the simplest: a hard cap on how many tokens the model can generate in a single response. Set it too low and you get truncated output — a response that just stops mid-thought because it hit the ceiling. Set it too high with no other constraint and you risk wasted compute on a generation that should have naturally ended much sooner (and, for API-billed models, wasted cost).

Stop sequences are a custom list of strings that immediately halt generation the instant they appear in the output. This is critical for structured outputs like JSON — you can set the stop sequence to the closing brace pattern you expect, or to a marker like '\n\n', preventing the model from generating unwanted spillover text after the actual answer is already complete. This lets you enforce strict response boundaries without needing heavy prompt engineering to ask the model nicely to stop.

Frequency penalty reduces the likelihood of reusing tokens that have already appeared frequently in the output so far — a positive value discourages repetition (useful for summarization, where restating the same point is wasted output), while a negative value actually exaggerates repetition (occasionally useful for something like poetry with intentional refrains).

Presence penalty is related but distinct: rather than tracking HOW OFTEN a token has appeared, it simply checks WHETHER a token has appeared at all, and pushes the model to bring in tokens it hasn't used yet. Higher values push harder for novelty; lower values let the model comfortably stick to patterns it's already established. This is handy for exploratory generation — brainstorming sessions, for instance — where you specifically want the model to range across a wide variety of ideas rather than circling back to the same few.

from openai import OpenAI

client = OpenAI()

# Structured extraction: cap length, hard-stop after the JSON closes,
# and keep repetition low since JSON keys naturally recur (default rate is fine)
resp = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "Extract name and age as JSON: 'Alex is 30.'"}],
    max_tokens=100,
    stop=["\n\n"],       # hard-stop right after the JSON object
    frequency_penalty=0,  # default — JSON syntax legitimately repeats
    presence_penalty=0,
)

# Brainstorm session: allow longer output, actively discourage repeating
# ideas, and push toward genuinely new ones each round
resp2 = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "Give me 10 startup ideas in EdTech."}],
    max_tokens=800,
    frequency_penalty=0.6,  # discourage restating the same phrasing
    presence_penalty=0.8,   # push toward genuinely new concepts, not variations
)