Re-ranking, Position Bias, Calibration, and Exploration
~45 min read
The final list adjustments that make rankings fair, calibrated, diverse, and safe for new content: position bias correction, calibration, MMR diversity, and exploration-vs-exploitation strategies.
Re-ranking, Position Bias, Calibration, and Exploration is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Position bias corrupts CTR training data — items at position 1 get more clicks regardless of quality; add position as a training feature and hold it constant at inference to debias
- •Calibration ensures predicted probabilities match actual event rates; miscalibrated scores cause incorrect ranking, bid pricing errors, and broken cross-model fusion
- •Exploitation-only systems create popularity feedback loops — add epsilon-greedy or UCB exploration from day one to ensure new content gets a fair chance