Re-ranking, Position Bias, Calibration, and Exploration

Re-ranking is the last stage before the list is shown to the user. It applies adjustments that the ranker model cannot express: diversity, freshness, business rules, bias correction, and exploration.

Position Bias

Items at position 1 receive far more clicks than items at position 5 — regardless of quality. This happens because users scan lists top-to-bottom and click the first plausible result. If you train a CTR model on data collected under position bias, the model learns to score position 1 highly, not quality.

Measuring position bias: fix items across positions in an A/B test and compare CTR at each position holding item quality constant. Typical finding: position 1 CTR is 3-5x higher than position 5 CTR for equivalent items.

Debiasing approaches:

1. Position as training feature: include position as a model input. At inference, set position=1 for all candidates — the model then predicts position-independent quality.

2. Inverse Propensity Scoring (IPS): weight each training example by 1/P(shown at position k) — rare positions get more weight.

# IPS weighted loss
position_propensity = {1: 0.5, 2: 0.3, 3: 0.15, 4: 0.05}  # empirical
weight = 1.0 / position_propensity[position]
loss = weight * binary_cross_entropy(y_true, y_pred)

3. Randomization: occasionally shuffle positions randomly and collect unbiased click data — expensive (lower short-term engagement) but gold-standard for training signal.

Calibration

A CTR model might consistently predict 8% for items that actually achieve 2% CTR. Miscalibration matters when:

• Fusing scores from multiple models (a 0.08 from model A and 0.08 from model B mean different things if models are miscalibrated differently)
• Using scores in bid optimization (ads systems charge based on predicted CTR)
• Making cross-category comparisons

Calibration check: divide predictions into deciles, plot mean prediction vs. mean actual CTR per decile — a perfectly calibrated model traces a diagonal.

Fix: Platt scaling (logistic regression on model output), isotonic regression (monotone calibration), or temperature scaling (divide logits by learned temperature T before softmax).

Exploration vs. Exploitation

Exploitation: always show the item with the highest predicted score. Problem: new items never get shown → no data collected on them → they score low forever (cold start + popularity feedback loop combined).

ε-greedy: with probability ε (e.g., 0.1), replace one item with a randomly selected unexplored item. Simple, interpretable, easy to implement.

UCB (Upper Confidence Bound): for item i, score = Q_i + C√(ln t / n_i), where Q_i = mean reward, n_i = number of times shown, t = total impressions. Automatically explores items with high uncertainty (low n_i).

Thompson Sampling: model each item's reward as a Beta distribution. Sample from each item's distribution. Show the item with the highest sample. Naturally balances exploration and exploitation and converges faster than UCB in practice.

Diversity (MMR)

Maximal Marginal Relevance prevents showing 10 nearly-identical items:

MMR_score(item_i) = λ × relevance(item_i) − (1 − λ) × max_j∈Selected sim(item_i, item_j)

At each step, greedily select the item maximizing MMR score. λ = 0.7: 70% weight on relevance, 30% on diversity. λ = 1.0: pure relevance (no diversity). λ = 0.0: maximum diversity.

Calibrate λ with A/B tests measuring both CTR (relevance) and scroll depth / session engagement (diversity).