advancedEvaluators — Testing & Safety
What is the specific risk that a 'RelevancyEvaluator' or similar check is designed to catch in a RAG system?
It catches cases where the retrieved context and the generated answer have drifted apart — the model answered something plausible-sounding but not actually grounded in (or even contradicting) the documents that were retrieved for it, which defeats the entire point of doing retrieval in the first place.
Ready to master this question?
Generate a complete walkthrough — background, the full answer in plain language, a working code example explained line by line, a real-world scenario, common mistakes, and how this same question gets asked in different ways.
Sign in to generate a response