Tool Selection: How the LLM Decides When and Which Tool to Call

Once a tool is defined (the previous subtopic), a separate question determines whether it actually gets used well: how does the LLM decide WHEN to call a given tool, and WHICH tool among several available ones is the right fit for a specific request? The mechanism is more straightforward than it might seem, but getting it right in practice takes real care.

At the API level, every available tool's name, description, and input schema are included in the request sent to the model. The model reads the user's message alongside this list of available tools and their descriptions, and decides — based purely on how well a tool's description matches what accomplishing the current request seems to require — whether to call a tool at all, and if so, which one. This means tool selection quality is almost entirely a function of description quality: a vague description ('handles currency stuff') gives the model much weaker signal than a specific one ('converts an amount from one currency to another using live exchange rates — use this whenever the user asks about currency conversion').

This connects directly to a broader principle covered elsewhere in this curriculum's agent-patterns content: more tools does not mean better results. Adding unnecessary tools — a speech-to-text module for a text-only task, a code execution environment nobody needs — can confuse the model's selection process and reduce efficiency, since a larger tool list with overlapping or vague descriptions makes the 'which tool fits this request' decision genuinely harder for the model to get right, not easier.

Practical guidance for reliable tool selection: keep each tool's description specific about WHEN to use it (not just what it does), avoid description overlap between multiple tools that could plausibly handle the same request, and keep the active tool list scoped to what's genuinely relevant for the current context rather than exposing every tool the application has ever defined on every single request.

from openai import OpenAI

client = OpenAI()

# Vague description — weak signal for tool selection
weak_tool = {
    "type": "function",
    "function": {
        "name": "currency_tool",
        "description": "Handles currency stuff",  # vague — poor selection signal
        "parameters": {"type": "object", "properties": {"query": {"type": "string"}}},
    },
}

# Specific description — clear signal for WHEN to use it
strong_tool = {
    "type": "function",
    "function": {
        "name": "convert_currency",
        "description": (
            "Converts an amount from one currency to another using live "
            "exchange rates. Use this whenever the user asks about currency "
            "conversion, exchange rates, or 'how much is X in Y currency'."
        ),
        "parameters": {
            "type": "object",
            "properties": {
                "amount": {"type": "number"},
                "source_currency": {"type": "string"},
                "target_currency": {"type": "string"},
            },
            "required": ["amount", "source_currency", "target_currency"],
        },
    },
}

resp = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "How much is 100 USD in EUR?"}],
    tools=[strong_tool],  # the specific description makes selection reliable
)