Key Differences: When Agentic Retrieval Beats Manual RAG, and When It Doesn't

The previous two subtopics laid out both approaches concretely: the manual pipeline (embed once, retrieve from one static source) and the agentic system (three layers, dynamically deciding what to search across multiple live sources). This subtopic makes the decision explicit — which approach fits which situation.

Manual RAG remains the right choice when your data lives in ONE place (or a small, fixed set of places you can reasonably combine into one vector store), doesn't change often relative to how it's queried (a policy handbook updated quarterly, not a live chat stream), doesn't need different access rules per user (everyone querying sees the same underlying documents), and where the query itself maps cleanly onto 'find similar text' (a factual lookup, not a query that implicitly spans multiple different KINDS of records). This describes a large fraction of genuinely useful RAG applications — internal documentation search, customer-facing FAQ bots grounded in a product manual, and similar single-corpus use cases are all comfortably within manual RAG's sweet spot, and reaching for the heavier agentic architecture for these would be needless over-engineering.

Agentic context retrieval becomes necessary specifically when this course's Chicago-office example pattern applies: the query implicitly spans multiple DIFFERENT systems (not just multiple documents within one system), those systems change frequently enough that staleness is a real concern (a calendar or chat history from last week may already be outdated), and/or different users are authorized to see different subsets of the underlying data (a manual pipeline's single shared vector store has no natural way to enforce per-user access control at retrieval time). Enterprise 'ask anything about our company's information' assistants — spanning email, chat, project trackers, and documents simultaneously — are the canonical case this architecture targets.

This course's own real-world evidence for where the line actually falls: this is precisely how giants like Google (in Vertex AI Search), Microsoft (in M365 products), AWS (in Amazon Q Business), etc., are solving it — these are specifically ENTERPRISE, multi-source, access-controlled products, not general-purpose single-document Q&A tools, which is exactly the category of problem the previous subtopic's three layers were built to address. The practical decision rule this suggests: default to manual RAG, and only reach for the substantially heavier agentic architecture once you can point to a genuine multi-source, freshness-sensitive, or access-controlled requirement that manual RAG structurally can't satisfy — not because agentic sounds more sophisticated.

# A decision helper capturing the book's own dividing line between
# manual RAG and agentic context retrieval.

def choose_retrieval_architecture(
    num_distinct_systems: int,
    data_changes_frequently: bool,
    needs_per_user_access_control: bool,
) -> str:
    """Reproduces the book's implicit decision rule: default to manual
    RAG; only justify the heavier agentic architecture with a genuine
    multi-source, freshness, or access-control requirement."""
    reasons_for_agentic = []
    if num_distinct_systems > 1:
        reasons_for_agentic.append("spans multiple distinct systems (not just documents)")
    if data_changes_frequently:
        reasons_for_agentic.append("sources change frequently -- staleness risk")
    if needs_per_user_access_control:
        reasons_for_agentic.append("needs per-user authorization at retrieval time")

    if reasons_for_agentic:
        return f"Agentic context retrieval needed. Reasons: {reasons_for_agentic}"
    return "Manual RAG is sufficient -- single static source, no access-control needs"

scenarios = [
    {"num_distinct_systems": 1, "data_changes_frequently": False,
     "needs_per_user_access_control": False},   # internal doc search
    {"num_distinct_systems": 4, "data_changes_frequently": True,
     "needs_per_user_access_control": True},    # the Chicago-office example
]

for s in scenarios:
    print(f"{s}\n  -> {choose_retrieval_architecture(**s)}\n")