Inside G-Eval: Auto-Generated Chain-of-Thought and Probability-Weighted Scoring

This curriculum's llm-evaluation topic already covers the big-picture shape of LLM-as-judge: direct/pointwise scoring (the 'G-Eval style') versus pairwise comparison (the 'Arena style'), plus the standard bias types (position, verbosity, self-preference, prompt sensitivity) and how to mitigate them. This subtopic goes one level deeper into G-Eval SPECIFICALLY, since its real mechanism is more interesting — and more useful to understand precisely — than 'ask the model to think step by step, then give a score.'

The first genuinely distinctive piece is auto-generated chain-of-thought (auto-CoT). Rather than a human writing out the specific reasoning steps a judge should follow for every new evaluation task, G-Eval takes your high-level evaluation CRITERIA (e.g. 'rate this summary's coherence from 1 to 5') and has the LLM itself generate a detailed, task-specific sequence of intermediate evaluation steps to follow — effectively having the judge model write its OWN detailed evaluation rubric from a short high-level description, rather than requiring a human to manually author detailed step-by-step instructions for every distinct criterion you want to measure. This auto-CoT step happens ONCE per criterion (not per example being judged), and the resulting detailed steps are then reused as part of the prompt for every actual evaluation using that criterion.

The second, more technically distinctive piece is probability-weighted scoring. A naive judge implementation asks the model to output a single score (say, an integer from 1 to 5) and just reads that printed number literally. G-Eval instead looks at the PROBABILITIES the model assigned to EACH possible score token (1, 2, 3, 4, 5) at the position where it outputs its score, and computes the final score as a WEIGHTED AVERAGE across all of them, using each score's output probability as its weight — rather than only trusting whichever single token happened to have the highest probability. Concretely: if the model assigns probability 0.1 to outputting '3', 0.6 to outputting '4', and 0.3 to outputting '5', the naive approach just reports '4' (the single most likely token); G-Eval's probability-weighted approach instead computes 0.1×3 + 0.6×4 + 0.3×5 = 4.2, capturing the model's genuine uncertainty between adjacent scores as a smooth, continuous number rather than forcing an artificial discrete choice.

Why this matters practically: this probability-weighting produces meaningfully more fine-grained, more stable scores than reading off a single discrete token, especially for borderline cases where the model is genuinely torn between two adjacent scores — those cases get a score that reflects the genuine uncertainty (like 4.2, hovering between a 4 and a 5) rather than an arbitrary coin-flip collapse to one integer or the other, which in turn makes aggregate statistics (like an average score across many examples) considerably less noisy than averaging discrete integer judgments would be.

# Implementing G-Eval's TWO distinctive mechanisms: auto-generated
# evaluation steps from a high-level criterion, and probability-
# weighted scoring across candidate score tokens.

def auto_generate_cot_steps(criterion: str) -> list[str]:
    """Stand-in for asking an LLM to expand a high-level criterion
    into detailed, task-specific evaluation steps -- done ONCE per
    criterion, then reused for every example judged against it."""
    templates = {
        "coherence": [
            "Read the summary and identify its main claims.",
            "Check whether claims are presented in a logical order.",
            "Check whether transitions between ideas are clear.",
            "Assign a coherence score from 1 (incoherent) to 5 (fully coherent).",
        ],
    }
    return templates.get(criterion, [f"Evaluate the text for {criterion}."])

def probability_weighted_score(score_token_probabilities: dict[int, float]) -> float:
    """G-Eval's core scoring mechanism: weight each POSSIBLE score by
    the probability the model assigned to it, instead of reading off
    only the single highest-probability token."""
    return sum(score * prob for score, prob in score_token_probabilities.items())

def naive_single_token_score(score_token_probabilities: dict[int, float]) -> int:
    """The naive alternative: just take the highest-probability token."""
    return max(score_token_probabilities, key=score_token_probabilities.get)

steps = auto_generate_cot_steps("coherence")
print("Auto-generated evaluation steps for 'coherence':")
for i, step in enumerate(steps, 1):
    print(f"  {i}. {step}")

# The model's output-token probabilities at the scoring position --
# a genuinely borderline case, torn between scores 4 and 5
borderline_case_probs = {3: 0.05, 4: 0.60, 5: 0.35}
print(f"\nNaive (single token) score:            {naive_single_token_score(borderline_case_probs)}")
print(f"G-Eval (probability-weighted) score:   {probability_weighted_score(borderline_case_probs):.2f}")
print("-> The weighted score (4.30) captures genuine uncertainty between")
print("   4 and 5, rather than forcing an artificial discrete choice")