Manual RAG vs. Agentic Context Retrieval: Building for Multi-Source Enterprise Data
Why naive 'embed it and RAG it' fails for real multi-source enterprise data, and the 3-layer Ingestion/Retrieval/Generation architecture (grounded in the open-source Airweave framework) needed to actually solve it.
Semantic paragraph chunking
PDF textbooks are parsed and split into chunks with 100-character overlaps to keep semantic continuity.
▶📚 Prerequisites(2)
🎓 Learning objectives
- •Explain why a query spanning multiple data sources (Linear, Calendar, Gmail, Slack) breaks naive single-source RAG
- •Describe the 3 layers of an agentic context retrieval system: Ingestion, Retrieval, and Generation
- •Identify the specific sub-problems each layer must solve (auth, per-source processing, incremental refresh, query expansion, source routing, hybrid search, authorization filtering, recency weighting)
- •Explain why detecting genuine content changes (vs. permission-only changes) is harder than simple timestamp comparison
What is it?
This topic covers the gap between 'toy RAG' (embed some static documents, do vector search) and what's actually required to build a unified query engine over real, messy, multi-source enterprise data — Gmail, Drive, Linear, Calendar, Slack, and more. Devs typically treat context retrieval like a weekend project: 'embed the data, store in a vector DB, do RAG' — this works beautifully for static, single sources, but no real-world workflow actually looks like this. Solving it properly requires thinking of context retrieval as building an Agentic Context Retrieval system with three critical layers: Ingestion, Retrieval, and Generation.
Why it exists
A single, simple example makes the gap concrete: 'What's blocking the Chicago office project, and when's our next meeting about it?' Answering this one query requires searching across Linear (for blockers), Calendar (for meetings), Gmail (for emails), and Slack (for discussions) — no naive RAG setup, built around one vector store over one source, can handle this. This topic exists because the jump from 'single-source RAG demo' to 'production multi-source enterprise context retrieval' is where most real implementations actually live, and it's a fundamentally different, much harder engineering problem than the demo suggests — described as 'months of engineering before your first query works.'
Problem it solves
It solves the specific failure mode where a team builds a working RAG demo over one document source, then discovers that real user queries need information spread across many disconnected systems with different auth models, different data shapes, different update frequencies, and different access-control rules. It also solves the 'how do I keep this fresh without re-embedding everything constantly' problem — naively re-syncing on a timer wastes enormous compute re-processing unchanged data, but detecting genuine changes is subtler than it looks (a source's last-modified timestamp can change due to a permission update, not actual content changes, so naive timestamp-based refresh can trigger unnecessary full re-embeds).
Intuition
Think of the difference between answering a question by consulting one well-organized reference book (single-source RAG) versus answering it by needing to check your email, your calendar, your project-tracker, and your team chat all at once, cross-referencing what each says (multi-source agentic retrieval). The second task isn't just 'four times harder' — it requires an entirely different kind of coordination: knowing which sources are even relevant to a given question, handling the fact that each source has its own login and format, and weighing which information is current versus stale when sources disagree.
Analogy
A naive single-source RAG setup is like hiring a librarian who's an expert on exactly one bookshelf — incredibly useful if your question is answerable from that shelf, useless the moment it isn't. Agentic context retrieval is like hiring a research coordinator who knows which of your building's twenty different departments (each with its own filing system, its own access badge requirements, its own way of organizing information) is relevant to a given question, knows how to request access to each, knows how to reconcile conflicting or outdated information between departments, and can compile a single coherent, sourced answer.
Technical explanation
The Ingestion layer must: connect to each application without forcing users through repeated, painful authentication flows for every source; process each different data source type appropriately before embedding, since email, code, and calendar data have fundamentally different structures and require different chunking/preprocessing strategies; and detect when a source has actually been updated so it can refresh embeddings incrementally rather than reprocessing everything, ideally without a full re-sync. This last point is subtler than it sounds: a naive approach compares timestamps to detect updates, but a changed timestamp doesn't necessarily mean the content changed — it might just mean a permission was updated — so timestamp-only detection risks unnecessary full re-embeds.
The Retrieval layer must: expand vague or underspecified queries to infer what the user actually wants (the Chicago-office example query doesn't explicitly say 'search Linear and Calendar and Gmail and Slack' — that routing has to be inferred); direct queries to the correct data sources rather than searching everything indiscriminately; layer multiple search strategies together, such as semantic search, keyword search, and graph-based search, since no single strategy is sufficient across heterogeneous source types; ensure retrieval only surfaces what the requesting user is actually authorized to see, respecting each source's own permission model; and weigh old versus new retrieved information appropriately — recent data generally matters more, but older context still counts and shouldn't be discarded outright.
The Generation layer must: provide a citation-backed LLM response, so the user can verify exactly which source (which Linear ticket, which email, which calendar event) backs each part of the answer, rather than an opaque, unverifiable synthesis.
This is precisely how large-scale enterprise search products solve the problem in practice — Google's Vertex AI Search, Microsoft's M365 products, and Amazon's Amazon Q Business all implement variations of this same 3-layer architecture. An open-source reference implementation of this exact pattern is Airweave, a fully open-source framework providing the context retrieval layer for AI agents across 30+ apps and databases — implementing authentication handling across apps, per-source data processing, multi-tool information gathering, recency weighting, update detection with real-time sync, and citation-backed response generation in the style of an answer engine like Perplexity.
Architecture
Three layers, each independently complex: Ingestion (per-source connectors + auth + format-specific processing + incremental-update detection), Retrieval (query expansion + source routing + multi-strategy search fusion + authorization filtering + recency weighting), and Generation (citation-backed synthesis). Unlike single-source RAG's linear pipeline (embed → store → retrieve → generate), this architecture has real branching and fan-out at the Ingestion layer (N separate source connectors) and fan-out/fan-in at the Retrieval layer (multiple search strategies run and merged per query), converging only at the final Generation step.
Workflow
- Before building anything, write out a handful of realistic user queries your system needs to answer, and for each, identify which sources it would actually need to span — this reveals whether you're facing a single-source or multi-source problem.
- For the Ingestion layer, build (or adopt) per-source connectors handling that source's specific auth flow and data format — don't assume one embedding pipeline works uniformly across email, code, and calendar data.
- Design update detection carefully: don't rely purely on timestamp comparison, since a timestamp change doesn't guarantee a content change (it might just be a permission update) — build in a way to distinguish genuine content changes from metadata-only changes to avoid unnecessary full re-embeds.
- For the Retrieval layer, implement query expansion (inferring what's actually being asked), source routing (searching only the relevant subset of sources per query), and multiple search strategies (semantic, keyword, graph-based) run together rather than relying on one strategy alone.
- Enforce authorization at retrieval time, not just at the UI layer — a result the requesting user isn't permitted to see should never even reach the ranking/synthesis stage.
- Implement recency weighting so newer information about a topic is favored, but design this as a weighting factor, not a hard cutoff, so relevant older context isn't discarded entirely.
- For the Generation layer, always attach citations to the specific source item (ticket, email, event, message) backing each part of the response, rather than producing an unverifiable, uncited synthesis.
- Consider adopting an existing framework (like Airweave) that has already solved many of these sub-problems across dozens of common app integrations, rather than building every connector and detection mechanism from scratch.
Example
Illustrative architecture sketch for the 3-layer system
SOURCE_CONNECTORS = { 'linear': LinearConnector(auth=oauth_linear), 'calendar': CalendarConnector(auth=oauth_calendar), 'gmail': GmailConnector(auth=oauth_gmail), 'slack': SlackConnector(auth=oauth_slack), }
def ingest(source_name: str): connector = SOURCE_CONNECTORS[source_name] for item in connector.fetch_changed_items(): # don't trust timestamp alone — verify content actually changed if connector.content_hash(item) != store.get_last_hash(item.id): store.upsert(connector.process(item)) # source-specific chunking
def retrieve(query: str, user) -> list[dict]: expanded = expand_query(query) # infer intent relevant_sources = route_to_sources(expanded) # e.g. ['linear', 'calendar', 'slack'] results = [] for source in relevant_sources: results += semantic_search(source, expanded) results += keyword_search(source, expanded) results = [r for r in results if user.is_authorized(r)] # enforce ACLs return rerank_with_recency_weighting(results)
def generate(query: str, results: list[dict], user) -> dict: context = format_with_citations(results) response = llm_complete(query, context) return {'answer': response, 'citations': [r.source_link for r in results]}
'What's blocking the Chicago office project, and when's our next meeting about it?'
results = retrieve(query, current_user) answer = generate(query, results, current_user)
answer['citations'] -> [linear.co/TICKET-123, calendar event link, slack thread link]
Real-world usage
Google's Vertex AI Search, Microsoft's M365 Copilot, and Amazon's Amazon Q Business are all large-scale, production implementations of exactly this 3-layer agentic context retrieval architecture, each solving authentication, per-source processing, and citation-backed generation across dozens of enterprise data sources at scale. Airweave is a 100% open-source framework implementing this same pattern across 30+ apps and databases, providing an accessible reference implementation for teams that want to build similar multi-source retrieval without starting from zero or committing to a single vendor's enterprise search product. Any internal 'ask anything about our company' AI assistant at a mid-to-large company — spanning email, chat, project management, and document storage — is, whether the team realizes it explicitly or not, solving this exact 3-layer problem, and teams that skip straight to naive single-source RAG for this kind of assistant consistently discover the gap once real user queries start spanning multiple systems.
Trade-offs
Building genuine multi-source agentic context retrieval is described as 'months of engineering before your first query works' — a dramatically larger investment than a single-source RAG demo, which can be built in an afternoon. This investment is only justified when your actual user queries genuinely span multiple, heterogeneous sources, as the Chicago-office-project example does — for a genuinely single-source use case (e.g., 'answer questions about our product documentation'), naive single-source RAG remains the right, much cheaper choice. Adopting an existing framework like Airweave trades some architectural control and vendor/project dependency for a large reduction in the engineering investment needed to reach a working multi-source system.
Visual explanation
A 3-layer stack diagram feeding a running example query ('What's blocking the Chicago office project, and when's our next meeting about it?').
Layer 1 (bottom): [Ingestion Layer] — separate connectors to [Linear] [Calendar] [Gmail] [Slack], each handling its own auth, its own data-format processing (email vs. code vs. calendar events are structurally different), and incremental refresh detection.
Layer 2 (middle): [Retrieval Layer] — the incoming query gets expanded/clarified, routed to the relevant subset of sources (not all 4 necessarily), searched using multiple strategies (semantic + keyword + graph-based) in parallel, filtered by what the requesting user is actually authorized to see, and weighted by recency (newer Slack messages about the blocker matter more than an old one, but old context isn't discarded entirely).
Layer 3 (top): [Generation Layer] — synthesizes a single citation-backed response referencing the specific Linear ticket, Calendar event, and Slack thread that answered the question.
Advantages
- —
Correctly handles real-world queries that naturally span multiple data sources, which naive single-source RAG cannot answer at all
- —
The 3-layer separation (Ingestion/Retrieval/Generation) gives a clear architecture for reasoning about and improving each concern independently
- —
Citation-backed generation makes answers verifiable, which matters enormously for enterprise trust and adoption
- —
Following the same pattern used by Google, Microsoft, and Amazon's enterprise search products means you're building toward a proven, battle-tested architecture rather than an unvalidated custom design
Disadvantages
- —
Represents a dramatically larger engineering investment than single-source RAG — described as months of work before the first query even functions
- —
Requires solving genuinely hard sub-problems (cross-app auth, incremental update detection distinguishing content vs. metadata changes, authorization-aware retrieval) that have no simple off-the-shelf solution unless adopting a framework
- —
Unnecessary complexity and cost for use cases that are genuinely single-source, where naive RAG already works fine
- —
Multi-source systems have more moving parts and more potential failure points (any one source's auth or format changes can break ingestion) than a single-source pipeline
Common mistakes
- —
Starting a multi-source enterprise assistant project with naive single-source RAG ('just embed everything and vector search it'), only discovering the architecture doesn't scale once real cross-source queries arrive
- —
Relying purely on timestamp comparison to detect source updates, triggering unnecessary full re-embeds when only a permission (not the actual content) changed
- —
Enforcing authorization only at the UI/display layer instead of at retrieval time, risking unauthorized content reaching the ranking or synthesis stage
- —
Using a single search strategy (usually just semantic/vector search) across all source types, when different sources and query types benefit from keyword or graph-based search as well
- —
Generating responses without citations, making it impossible for users to verify which source backs a given claim — a serious trust problem for enterprise use cases
📂 Subtopics
The Manual RAG Pipeline: 'Embed It and RAG It' — and Where It Works
The book's framing of the naive default approach to context retrieval: embed the data, store it in a vector DB, do RAG. It works beautifully for static sources — the next subtopic covers exactly where it doesn't.
~11 min
Agentic Context Engineering: The Multi-Source Query Problem
The book's concrete motivating example — a single query spanning Linear, Calendar, Gmail, and Slack — shows exactly why naive RAG breaks down, and introduces the 3-layer agentic solution.
~13 min
Key Differences: When Agentic Retrieval Beats Manual RAG, and When It Doesn't
Manual RAG and agentic context retrieval solve different-shaped problems. Knowing which shape your problem has — single static source vs. multi-source, dynamic, access-controlled data — is the real decision to make.
~12 min
Infrastructure Requirements: Ingestion, Retrieval, and Generation Layers (Airweave)
The book's full breakdown of what each of the 3 layers actually requires — and Airweave, its named open-source reference implementation across 30+ apps and databases.
~14 min