Level 3 — Tool Calling: The LLM Decides When and How to Act

Level 3, Tool Calling, extends the Router pattern's 'pick from a known set' idea into genuine action-taking. A human defines a set of tools the LLM can access to complete a task — a weather API, a calculator, a database query function, a search tool — but crucially, the LLM now decides WHEN to use them, not just which pre-defined path applies to a static input. It also decides the actual arguments for execution — not just 'call the weather tool' but 'call the weather tool with the argument "Tokyo"', determined from the actual content of the request.

This is a genuine step up from Level 2's routing: a router picks one path from a fixed menu based on classifying the input; tool calling involves the LLM reasoning about what action would actually help accomplish the task, deciding whether that action is even necessary at all (many requests need zero tool calls), and constructing the specific arguments that action needs — none of which is a simple classification decision the way routing is.

A concrete example: given the request 'What's the weather like in Tokyo right now, and should I bring an umbrella?', a Level 3 system recognizes it needs current weather data (a fact the model's training data can't provide), decides to call a weather tool, constructs the argument 'Tokyo' from the request, executes the call, and then uses the tool's actual result to answer the umbrella question — genuine reasoning about whether and how to act, not just picking a lane.

The human's role at this level shifts to defining the available toolset and each tool's interface (what arguments it takes, what it returns) rather than defining the full decision logic — the LLM fills in the actual when-and-how decisions at runtime. This is the level most people mean when they informally say 'the LLM can use tools,' and it's the foundation the higher levels (multi-agent, autonomous) build further capability on top of.

from openai import OpenAI
import json

client = OpenAI()

def get_weather(city: str) -> str:
    # Stand-in for a real weather API call
    return f"{city}: 22C, light rain expected this afternoon"

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

def level3_tool_calling(user_message: str) -> str:
    """Level 3: the LLM decides IF a tool is needed, WHICH tool, and
    WHAT arguments to pass — all inferred from the request itself."""
    messages = [{"role": "user", "content": user_message}]
    resp = client.chat.completions.create(model="gpt-4.1", messages=messages, tools=tools)
    choice = resp.choices[0].message

    if choice.tool_calls:
        call = choice.tool_calls[0]
        args = json.loads(call.function.arguments)  # LLM-constructed arguments
        tool_result = get_weather(**args)
        messages += [choice, {"role": "tool", "tool_call_id": call.id, "content": tool_result}]
        final = client.chat.completions.create(model="gpt-4.1", messages=messages)
        return final.choices[0].message.content

    return choice.content  # no tool call needed for this request

print(level3_tool_calling("What's the weather in Tokyo, and should I bring an umbrella?"))