Agentic RAG vs. Traditional RAG: A Decision Framework

Every subtopic so far has described what agentic RAG adds over naive RAG — but 'more robust' isn't automatically 'the right choice,' since every one of those additions (decision agents, source selection, relevance checking, retry loops) costs extra LLM calls, extra latency, and extra engineering complexity to build and maintain. A practical decision framework matters as much as understanding the mechanics.

Reach for naive RAG when: your queries are consistently simple and fact-based, where the question and answer are semantically close enough that direct similarity search reliably works; your latency budget is tight (every extra agent call adds real round-trip time); you have a single, well-curated knowledge source rather than multiple sources to choose between; and you haven't yet observed a specific, repeatable failure mode that a more complex architecture is designed to fix. This lines up with the general guidance elsewhere in this curriculum: start at naive RAG as your baseline, and move to something fancier only once you hit a documented failure a specific architecture solves.

Reach for agentic RAG when: queries vary widely in whether they even need retrieval at all (some can be answered directly, wasting retrieval cost on naive RAG every time); you have multiple candidate sources (vector DB, live APIs, web search) and the right one genuinely depends on the specific query; you're seeing naive RAG return responses that don't actually address the question, with no way to detect or recover from this automatically; or your queries commonly require multi-step reasoning or decomposition that a single retrieval pass structurally can't handle.

A useful middle-ground worth knowing: you don't have to choose one architecture for an entire system. It's common in production to route different query types differently — a fast naive-RAG path for the bulk of simple queries, escalating to the full agentic pipeline only for queries that a lightweight classifier flags as complex, ambiguous, or requiring multiple sources. This captures most of naive RAG's speed for the common case while still getting agentic RAG's robustness exactly where it's actually needed.

from openai import OpenAI

client = OpenAI()

def classify_query_complexity(query: str) -> str:
    """Cheap upfront routing decision — a lightweight classifier, not a
    full agentic pipeline, deciding which pipeline to actually invoke."""
    resp = client.chat.completions.create(model="gpt-4.1", messages=[
        {"role": "user", "content":
            f"Classify this query as 'simple' (one clear fact lookup) or "
            f"'complex' (multi-part, ambiguous, or needs multiple sources): {query}"}
    ])
    return "complex" if "complex" in resp.choices[0].message.content.lower() else "simple"

def routed_rag(query: str, vector_store) -> str:
    complexity = classify_query_complexity(query)

    if complexity == "simple":
        return naive_rag(query, vector_store)          # fast path — most queries
    return iterative_agentic_rag(query, vector_store)  # full pipeline — only when needed

# naive_rag() and iterative_agentic_rag() as defined in the earlier subtopics