advancedEvaluators — Testing & Safety

Why can't you just use normal unit-test assertions (`assertEquals`) to test LLM output quality?

LLM output is non-deterministic and phrased differently each time even when factually correct, so exact string matching almost always fails even for a 'correct' answer. Evaluators instead use techniques like using a second LLM call to judge whether the output meets criteria (relevance, factual grounding, absence of the actual answer inside the prompt/leakage), producing a score rather than a pass/fail on exact text.

Ready to master this question?

Generate a complete walkthrough — background, the full answer in plain language, a working code example explained line by line, a real-world scenario, common mistakes, and how this same question gets asked in different ways.

Sign in to generate a response

Next Step

Continue to What is the specific risk that a 'RelevancyEvaluator' or similar check is designed to catch in a RAG system?← Back to all Spring AI questions