Cold Start, Evaluation, and System Tradeoffs
~40 min read
Solving cold start for new users and new videos, designing the A/B testing framework, closing the feedback loop, and navigating the key system tradeoffs.
Cold Start, Evaluation, and System Tradeoffs is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Cold start for new users: onboarding topic selection → demographic proxy embedding → popularity fallback → fast transition to personalized model after 5-10 interactions
- •Cold start for new videos: content-based embedding at upload + explicit exploration injection to collect initial engagement data
- •A/B tests for recommendation need at least 2 weeks and must track guardrail metrics (7-day retention, diversity) alongside the primary metric to catch short-term vs. long-term tradeoffs