Agentic RAG: Letting the LLM Decide When and What to Retrieve
~15 min read
Agentic RAG uses AI agents with planning, reasoning, and memory to orchestrate retrieval — deciding IF retrieval is needed, WHICH source to query, and validating the results, with a retry loop back to the start when the answer isn't good enough.
Agentic RAG: Letting the LLM Decide When and What to Retrieve is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Naive RAG's root problem is zero decision-making — Agentic RAG fixes this with planning/reasoning/memory-equipped agents orchestrating retrieval
- •A decision agent first checks IF retrieval is even needed, rather than always retrieving
- •A source-selection agent picks WHICH source to query (vector DB, tools/APIs, web) instead of a single fixed source
- •A relevance-checking agent validates the final answer and can loop the whole process back if it isn't good enough, up to a bounded iteration count
- •More robust than naive RAG at every step, at the cost of real added complexity and latency from multiple extra LLM calls per query