Practical Tuning Guide: Which Parameters to Change for Which Use Case

With 7 parameters (plus decoding strategy) to choose from, the practical question engineers actually face is: given a specific task, which of these actually matter, and which direction do I turn them? Three common use cases illustrate the pattern.

Creative writing (stories, marketing copy, brainstorming): favor diversity. Temperature 0.8-1.0, top-p around 0.9-0.95 (or top-k in the 40-100 range), a moderate presence penalty (0.3-0.6) to push toward fresh ideas rather than circling the same ones, and multinomial sampling rather than greedy or beam search — you WANT the unpredictability. Max tokens should be generous since you're not trying to bound the output tightly. Frequency penalty can stay low unless you notice specific phrase repetition creeping in.

Factual QA / chatbots (support bots, document Q&A, RAG-backed assistants): favor reliability over variety. Temperature low (0.0-0.3), top-p tight (0.85-0.9) or omitted entirely at very low temperature (it barely matters once temperature is near-deterministic), presence and frequency penalties near 0 (you don't want the model avoiding correct terminology just because it already used it once), and greedy or near-greedy multinomial decoding. Stop sequences matter here if you're extracting a bounded answer rather than open-ended prose.

Code generation: closer to factual QA than creative writing, since correctness matters far more than stylistic variety, but with a couple of specific twists. Temperature low (0.0-0.2) for most code — you want the statistically most likely (and thus usually most idiomatic and correct) token sequence, not a creative one. Stop sequences are especially important here (e.g. stopping at a closing code fence or a specific delimiter) to avoid the model rambling past the actual code block. Max tokens should be sized to the expected code length with headroom, since truncated code is often worse than no code at all. Beam search is sometimes used for code specifically because correctness of the full sequence (does it actually compile/run) matters more than any single token choice.

The general pattern across all three: temperature and sampling strategy are your primary diversity dial, penalties are secondary fine-tuning once the primary dial is roughly right, and stop sequences plus max tokens are about boundary control rather than creativity at all — get those right regardless of which use case you're in.

# A practical parameter preset table, ready to plug into an API call.
PRESETS = {
    "creative_writing": {
        "temperature": 0.9, "top_p": 0.95,
        "frequency_penalty": 0.2, "presence_penalty": 0.5,
        "max_tokens": 1200,
    },
    "factual_qa": {
        "temperature": 0.1, "top_p": 0.85,
        "frequency_penalty": 0.0, "presence_penalty": 0.0,
        "max_tokens": 400,
    },
    "code_generation": {
        "temperature": 0.15, "top_p": 0.9,
        "frequency_penalty": 0.0, "presence_penalty": 0.0,
        "max_tokens": 800, "stop": ["```"],
    },
}

from openai import OpenAI
client = OpenAI()

def generate(prompt: str, use_case: str) -> str:
    params = PRESETS[use_case]
    resp = client.chat.completions.create(
        model="gpt-4.1",
        messages=[{"role": "user", "content": prompt}],
        **params,
    )
    return resp.choices[0].message.content

print(generate("Write a tagline for a coffee brand.", "creative_writing"))
print(generate("What is the capital of France?", "factual_qa"))