Level 2 — Router Pattern: The LLM Picks a Path

Level 2, the Router pattern, is where the LLM first gains any real decision-making role, even though that role is still tightly bounded. A human defines the paths or functions that exist in the flow in advance — a fixed, known set of options — and the LLM's job is to make a basic decision about which one of those pre-defined paths applies to a given input.

Concretely: imagine a customer support system with three possible handling paths — 'billing question,' 'technical issue,' or 'general inquiry' — each wired up to different downstream logic (different prompts, different tools, different escalation rules). At Level 1, a human would have to manually route every incoming message to the right path. At Level 2, the LLM reads the incoming message and picks which of the three pre-defined paths it belongs to — the human still built and defined all three paths ahead of time, but the LLM now decides which one gets used for any given input.

This is a meaningfully bigger step than it might first appear: the LLM is now influencing the actual program flow, not just producing content within a flow someone else has fully mapped out step by step. But the LLM's decision space is still tightly bounded — it can only choose among the paths a human already built, it can't invent a new path, call an arbitrary tool, or take a multi-step action beyond the single routing decision.

Router-pattern systems are extremely common in production precisely because they capture real value (letting the LLM handle the classification/routing decision that would otherwise need manual rules or a separate classifier) while keeping the blast radius of any single LLM mistake small and contained — a wrong routing decision sends a query down the wrong pre-built path, but it can't cause the system to do something genuinely unplanned or ungoverned.

from openai import OpenAI

client = OpenAI()

# Human-defined paths, decided and built in advance
PATHS = {
    "billing": lambda msg: f"[Billing team] Handling: {msg}",
    "technical": lambda msg: f"[Technical team] Handling: {msg}",
    "general": lambda msg: f"[General inquiries] Handling: {msg}",
}

def level2_router(message: str) -> str:
    """Level 2: the LLM picks WHICH pre-defined path to use — it can't
    invent a new path or take any action beyond this one routing choice."""
    resp = client.chat.completions.create(
        model="gpt-4.1",
        messages=[{"role": "user", "content":
            f"Classify this message as exactly one of: billing, technical, general.\n\n{message}"}],
    )
    chosen_path = resp.choices[0].message.content.strip().lower()
    handler = PATHS.get(chosen_path, PATHS["general"])  # fallback if unclear
    return handler(message)

print(level2_router("My credit card was charged twice this month."))