Prompting Techniques for Reliable JSON: Schema-in-Prompt & Few-Shot Examples

Getting reliable JSON out of an LLM isn't just about asking for JSON — it's about a few compounding techniques that each reduce a different source of failure.

The foundation is putting the exact schema directly in the prompt, spelled out field by field with explicit types — string, number, boolean, an array type, or an enum-style union like "low"|"medium"|"high". The more precisely you specify the shape, the less room the model has to improvise a different structure. This is the technique from the previous subtopic, and it's necessary but usually not sufficient on its own for edge cases.

Few-shot examples close the gap the schema alone leaves open. A schema tells the model WHAT shape to produce, but not always exactly HOW to handle ambiguous or edge-case inputs — what should 'key_points' look like when the source text has zero clear points? Should 'priority' default to 'low' or should the field be omitted? Showing 1-3 concrete input/output examples inside the prompt demonstrates the exact behavior you want in exactly the situations your schema description alone left ambiguous, which meaningfully reduces the variance in edge-case handling.

Beyond prompt-level techniques, most modern LLM APIs also offer a native structured-output mode — OpenAI's response_format={"type": "json_object"} or the stricter json_schema mode, Anthropic's structured tool-use pattern, and similar features elsewhere. These modes constrain the model's decoding process itself (not just the prompt) to only emit tokens that form valid JSON, which is a fundamentally stronger guarantee than prompting alone — a model can still ignore prompt instructions about format under prompting alone, but a properly-configured JSON mode structurally cannot emit invalid JSON at the decoding level.

The most reliable pipelines in practice stack all three: schema spelled out in the prompt (for content correctness — the model still needs to know what values to put where), a couple of few-shot examples for the trickiest edge cases in your domain, and native JSON/schema mode enabled at the API level as a structural backstop.

from openai import OpenAI

client = OpenAI()

SCHEMA_PROMPT = """
Extract fields as JSON matching this schema exactly:
{{"sender_intent": string, "key_points": string[], "priority": "low"|"medium"|"high"}}

Examples:
Input: "FYI, the deploy finished successfully."
Output: {{"sender_intent": "info", "key_points": [], "priority": "low"}}

Input: "URGENT: production database is down, need help NOW."
Output: {{"sender_intent": "request", "key_points": ["production db down"], "priority": "high"}}

Now extract from:
{email}
""".strip()

def extract_email(email: str) -> dict:
    resp = client.chat.completions.create(
        model="gpt-4.1",
        messages=[{"role": "user", "content": SCHEMA_PROMPT.format(email=email)}],
        response_format={"type": "json_object"},  # structural backstop
    )
    import json
    return json.loads(resp.choices[0].message.content)

print(extract_email("Reminder: standup moved to 10am tomorrow."))