Ranking: Feature Engineering and Learning-to-Rank

The ranker's job: score each candidate precisely using features that were too expensive to compute during retrieval. This is where most of the quality lift comes from.

Feature Categories

User features:

• Demographics: age, location, language, device
• Behavioral history: top categories, average session length, time-of-day patterns
• Recent signals: last 5 clicked items (real-time, from feature store)

Item features:

• Content: category, tags, duration, language
• Historical engagement: global CTR, average watch time, like rate
• Freshness: time since publication (newer items may get a boost)

Context features:

• Time of day, day of week
• Device type (mobile vs. desktop)
• Session context (what has the user done in this session so far?)

Cross Features (most powerful): Cross features capture interaction effects between user and item properties.

user_country × item_language   → local content preference
user_age_bucket × item_genre   → age-genre affinity
time_of_day × content_type     → morning news, evening entertainment
device_type × item_duration    → short videos on mobile, long videos on TV

A model trained only on user features and item features separately will miss these interactions. DCN (Deep & Cross Network) automates cross-feature generation.

Learning-to-Rank Losses

Pointwise: treat each item independently — predict P(click | user, item, context). Loss: binary cross-entropy on (item, clicked/not_clicked) pairs. Simple to implement; most production systems use this. Weakness: doesn't model the ranking relationship between items.

Pairwise: for each pair of items (i, j), predict which one the user prefers. Loss: binary cross-entropy on (item_clicked > item_not_clicked) pairs. Captures relative ordering but is quadratic in items per user.

Listwise: optimize ranking quality metrics (NDCG, MAP) directly over the full list. LambdaMART: gradient boosting with lambda gradients that weight pairs by NDCG change. Best aligned with actual ranking quality but hardest to implement.

Deep Ranking Models

Wide & Deep (Google Play, 2016):

Wide component: memorization
  Sparse cross-product features → linear model
  Captures 'users who buy butter also buy bread' type co-occurrences

Deep component: generalization
  Dense embeddings for user/item IDs → MLP
  Generalizes to unseen user-item pairs

Output: sigmoid(wide_output + deep_output) → P(click)

DIN (Deep Interest Network, Alibaba): Uses attention over the user's behavioral history to weight items relevant to the current candidate. If a user browsed 50 products, DIN attends to the 5 most similar to the candidate being scored. Captures: 'this user is currently in a shoe-browsing session' (vs. their all-time electronics interest).

Practical training setup:

1. Negative sampling: 1 positive click per 4-10 randomly sampled non-clicked items
2. Handle class imbalance: positive rate is 1-5% of impressions
3. Add position as training feature (critical for debiasing — see re-ranking section)
4. Evaluate with AUC, NDCG@10, and calibration plot (predicted vs. actual CTR)