Attentive Reasoning Queries (ARQ)

CoT, Self-Consistency, and Tree of Thoughts all improve how a model reasons through a problem, but they still rely on free-form thinking — and free-form thinking breaks down in long, rule-heavy tasks. Give an agent a 2,000-word system prompt full of policies, tone rules, and behavioral dos and don'ts, and you'd expect it to follow every word. In practice: it starts strong, then drifts and starts hallucinating, then forgets what was said five turns ago — and the agent that was told to 'never promise a refund' ends up happily offering one. Even CoT doesn't fully fix this, because the reasoning stays free-form — the model 'thinks aloud' but has limited domain-specific control over what it attends to.

Attentive Reasoning Queries (ARQs) solve this by replacing free-form reasoning with explicit, domain-specific questions. Instead of letting the model reason however it wants, each reasoning step is encoded as a targeted query inside a JSON schema. Before making a recommendation or a tool call, the model is prompted to fill structured keys — for example, 'does_policy_allow_refund: bool', 'user_explicitly_requested_refund: bool', 'alternative_offered: string|null' — rather than writing an open paragraph of reasoning.

This does two things: (1) it reinstates critical instructions by keeping the model aligned mid-conversation (the query itself re-surfaces the rule), and (2) it facilitates intermediate reasoning that is auditable and verifiable, since every step is a structured, checkable field rather than free prose. By the time the model generates its final response, it has already walked through a sequence of controlled reasoning steps with no free-text exploration — unlike CoT or Tree of Thoughts.

Across 87 test scenarios, ARQ reached a 90.2% success rate, versus 86.1% for CoT reasoning and 81.5% for direct response generation. ARQ is implemented in Parlant, an open-source framework for building instruction-following agents, where it's woven into three modules: the guideline proposer (deciding which behavioral rules apply), the tool caller (determining what external functions to use), and the message generator (producing the final customer-facing reply).

The core insight generalizes beyond any specific framework: when you make reasoning explicit, measurable, and domain-aware, LLMs stop improvising and start reasoning with intention. Free-form thinking sounds powerful, but in high-stakes or multi-turn scenarios, structure wins.

# Example ARQ schema for a customer-support refund decision
ARQ_SCHEMA = {
    "type": "object",
    "properties": {
        "policy_rule_recalled": {
            "type": "string",
            "description": "State the exact refund policy rule that applies here",
        },
        "user_explicitly_requested_refund": {"type": "boolean"},
        "order_within_refund_window": {"type": "boolean"},
        "alternative_resolution_offered": {
            "type": ["string", "null"],
            "description": "e.g. store credit, replacement — null if none offered",
        },
        "final_decision": {
            "type": "string",
            "enum": ["approve_refund", "deny_refund", "escalate_to_human"],
        },
    },
    "required": [
        "policy_rule_recalled", "user_explicitly_requested_refund",
        "order_within_refund_window", "final_decision",
    ],
}
# Forcing the model to fill `policy_rule_recalled` before `final_decision`
# re-surfaces the relevant policy on every turn, preventing the multi-turn
# drift that plain CoT is prone to.