Compression and Selection: Filtering Aggregated Context Before Generation
~12 min read
The book's step #7, filter context, uses a dedicated context-evaluation agent to trim the combined output of 4 retrieval sources before it ever reaches the response-generating agent.
Compression and Selection: Filtering Aggregated Context Before Generation is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Step #7 passes the combined 4-source context to a dedicated context-evaluation agent that filters out irrelevant context before generation
- •Filtering is done by an LLM-powered agent, not brittle rules, because judging relevance is nuanced — better suited to LLM judgment
- •Filtering is a separate step decoupled from response generation, keeping the context-evaluation agent's job narrow: decide what stays, not write the answer
- •This is a distinct Compressing mechanism from summarization: it makes binary keep/drop decisions on whole items rather than shrinking each item's token count
- •Narrowing context before generation controls both token cost and the risk of the final response being diluted or distracted by irrelevant retrieved material