Problem Framing and Feature Engineering
~40 min read
What CTR prediction is, why it's necessary for ad auctions, and the four feature categories that power a CTR model — with emphasis on cross-features as the highest-signal input type.
CTR Prediction in the Ad Ecosystem
Every ad-supported platform runs an ad auction at each impression opportunity. When a user loads a page with an ad slot, the platform must:
- Identify N candidate ads competing for this slot
- Predict P(click | user, ad, context) for each candidate
- Compute eCPM = predicted_CTR × bid_price × 1000 for each candidate
- Show the highest-eCPM ad
Without CTR prediction, all ads would have the same effective price regardless of relevance. The platform would show whichever ad bid the most, even if the user would never click it — destroying both user experience and advertiser ROI.
Why CTR prediction is harder than it looks:
- Class imbalance: ~2% of impressions result in clicks; 98% don't
- Sparsity: most (user, ad) pairs are never observed in training
- Scale: 1B+ impressions/day → trillions of training examples per year
- Latency: prediction must complete in < 10ms during the auction
- Calibration: predicted CTR must match actual click rate in absolute terms
Feature Engineering — The Four Categories
Category 1: User features
- Demographics: age_bucket, gender, country, language
- Device: device_type (mobile/desktop/tablet), OS, browser
- Behavioral history: past 7-day CTR by ad category, past ad formats clicked
- User embedding: dense learned representation from user's full history
Category 2: Ad features
- Identity: advertiser_id, campaign_id, ad_id (unique creative)
- Content: ad_category, ad_format (banner/video/native), ad_size
- Creative: title embedding (from NLP model), image embedding (from CNN)
- Historical performance: global CTR for this ad (avg across all past impressions)
Category 3: Context features
- Platform: web vs. mobile app vs. tablet app
- Page category: what content is the user currently viewing?
- Time: hour_of_day, day_of_week (CTR follows strong temporal patterns)
- Slot: position of the ad on the page (first slot has higher CTR)
Category 4: Cross-features (highest signal for CTR) Cross-features capture interactions between the above categories:
| Cross-feature | Why it matters |
|---|---|
| user_country × ad_language | English ad to French user → very low CTR |
| user_device × ad_format | Video ad on slow connection → low CTR |
| page_category × ad_category | Sports page + sports equipment → high CTR |
| user_age_bucket × ad_category | Teen + gaming = different from adult + gaming |
| user_app_history × ad_category | User installed fitness apps → fitness ad |
The most powerful sparse cross-feature: user_app_history × ad_id (which apps has this user installed historically × which specific ad is shown) This creates billions of unique cross-feature values but captures direct 'user type A clicked ad B in the past → likely to click again.'
This is the cross-feature the Wide component of Wide & Deep is designed to memorize.
💬 Deep Dive with AI
Key points
- •CTR prediction is binary classification (click/no-click) at 1B+ impressions/day with the additional requirement of calibrated probabilities (for auction pricing) beyond just ranking quality
- •Cross-features (interactions between user, ad, and context features) consistently carry more predictive signal than any individual feature category — especially sparse cross-features like user_app_history × ad_category
- •Class imbalance (~2% click rate) requires negative downsampling in training, which introduces calibration bias that must be explicitly corrected post-training