Why JSON Prompting Matters: The Structured Output Problem

When you give an LLM an open-ended instruction — 'summarize this email and give me the key points' — it has to guess what 'good output' actually looks like for you. Sometimes it adds extra commentary you didn't ask for. Sometimes it skips details you actually needed. Sometimes the formatting changes between one call and the next for no discernible reason. The problem here isn't that the model is bad — it's that the prompt itself left too much room for interpretation.

Natural language is powerful precisely because it's flexible, but that flexibility is exactly the liability for tasks like extraction, reporting, automation, or analysis, where you need the output to stay consistent every single time it runs. A vague instruction like 'summarize this' or 'give me key takeaways' leaves gray areas that can lead to hallucinations or format drift, because the model has to fill in unstated assumptions about exactly what you want.

JSON prompting solves this by specifying the exact schema you want the output to fill in, rather than describing what you want in open-ended prose. Instead of 'summarize this email and give me the key points,' you write something like: 'Extract the following as JSON: {sender_intent: string, key_points: string[], action_required: boolean, priority: "low"|"medium"|"high"}.' There's no more ambiguity about what fields should exist or what type each one should be.

The reason this works so well isn't a coincidence — AI models are trained on massive amounts of structured data from APIs and web applications, so JSON is effectively closer to their 'native language' than loosely-specified natural instructions are. When you speak that native language back to the model, it tends to respond with noticeably more precision and consistency than an equivalent prose instruction would produce.

from openai import OpenAI

client = OpenAI()

email = "Hey team, the client wants the invoice resent by Friday, urgent."

# Vague, open-ended instruction — output shape is unpredictable
vague_prompt = f"Summarize this email and give me the key points: {email}"

# JSON-prompted instruction — output shape is guaranteed by the schema
json_prompt = f"""
Extract the following as JSON from this email. Match this exact schema:
{{"sender_intent": string, "key_points": string[], "action_required": boolean, "priority": "low"|"medium"|"high"}}

Email: {email}
""".strip()

resp = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": json_prompt}],
    response_format={"type": "json_object"},
)
print(resp.choices[0].message.content)
# {"sender_intent": "request", "key_points": ["resend invoice"],
#  "action_required": true, "priority": "high"}