Corrective RAG (CRAG): Self-Evaluating Retrieval

Naive RAG has no mechanism to notice when its own retrieval step returned bad results — whatever gets retrieved gets passed straight to the LLM, correct or not. Corrective RAG (CRAG) directly targets this gap by validating retrieved results before trusting them, comparing them against trusted sources such as a live web search, and either filtering out or correcting retrieved content before it ever reaches the generation step.

The core addition over naive RAG is a relevance-evaluation stage sitting between retrieval and generation. After the initial vector search returns candidate documents, an evaluator (often a lightweight classifier or the LLM itself acting as a judge) scores how relevant and trustworthy each retrieved chunk actually is to the query — not just how high its cosine similarity was, but whether it genuinely contains information that answers the question.

Based on that evaluation, CRAG branches: if the retrieved content scores as sufficiently relevant and trustworthy, the pipeline proceeds like naive RAG, generating directly from what was retrieved. If the content scores as insufficient, ambiguous, or outdated, the system falls back to an external, trusted source — commonly a live web search — to pull in fresher or more reliable information instead of generating from a weak vector-store match. This is especially valuable for queries touching fast-changing information (current events, recent product specs, pricing) where a vector store's static, previously-indexed content can be stale even when it's technically the closest semantic match available.

The practical effect is a RAG pipeline that's meaningfully more robust to two of naive RAG's biggest weaknesses at once: it catches irrelevant or low-quality retrieval instead of blindly trusting cosine similarity, and it has a real fallback path (external search) for exactly the cases where the vector store alone isn't good enough — at the cost of an extra evaluation step (and sometimes a web search call) added to the pipeline's latency.

from openai import OpenAI

client = OpenAI()

def evaluate_relevance(query: str, chunk: str) -> float:
    """LLM-as-judge relevance score in [0, 1] — a real system might use
    a lighter, faster classifier here instead of a full LLM call."""
    resp = client.chat.completions.create(
        model="gpt-4.1",
        messages=[{"role": "user", "content":
            f"On a scale 0-1, how relevant is this passage to the query?\n"
            f"Query: {query}\nPassage: {chunk}\nRespond with only a number."}],
    )
    return float(resp.choices[0].message.content.strip())

def corrective_rag(query: str, vector_store, web_search, threshold: float = 0.6) -> str:
    results = vector_store.similarity_search(query, k=3)
    scored = [(r, evaluate_relevance(query, r.text)) for r in results]
    good_results = [r for r, score in scored if score >= threshold]

    if good_results:
        context = "\n\n".join(r.text for r in good_results)
    else:
        # Fallback: vector store results weren't good enough — go external
        context = web_search(query)

    resp = client.chat.completions.create(
        model="gpt-4.1",
        messages=[{"role": "user", "content": f"Context:\n{context}\n\nQuestion: {query}"}],
    )
    return resp.choices[0].message.content