Wide & Deep Architecture and Model Evolution
~45 min read
The evolution from logistic regression to GBDT+LR to Wide & Deep to attention-based models, with a deep dive on what each component contributes and why joint training matters.
Wide & Deep Architecture and Model Evolution is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •GBDT+LR's key insight: gradient boosted trees automatically discover feature transformations and interactions that manual feature engineering misses — then LR combines them
- •Wide & Deep formally separates memorization (wide: logistic regression on sparse cross-features) from generalization (deep: MLP on embeddings) — joint training allows each component to specialize while the combined model benefits from both
- •DIN's key advance over W&D: attention over user history weighted by relevance to the current ad gives a dynamic user representation instead of a static average of all past interactions