Candidate Generation: Two-Tower Models and Collaborative Filtering

Candidate generation is Stage 1 of the recommendation funnel. Its job: retrieve a manageable set of plausible candidates from a catalog that's too large to score exhaustively. Optimize for recall — missing a great item here means it can never be recommended.

Two-Tower Model

Architecture: two independent neural networks — one for users, one for items — each producing an embedding vector of the same dimension.

User Tower inputs:          Item Tower inputs:
  user_id embedding            item_id embedding
  age, country                 category, tags
  watch_history_emb            avg_rating, views
         ↓                            ↓
   Dense layers                Dense layers
         ↓                            ↓
  user_emb (128d)            item_emb (128d)
         ↓                            ↓
         └──── dot_product ────────────┘
                     ↓
             relevance score

Training uses in-batch negatives: for each (user, item) positive pair in the batch, use all other items in the batch as negatives. Efficient — no separate negative mining pass.

Serving: pre-compute all item embeddings → build FAISS HNSW index. At query time, compute user embedding → approximate nearest neighbor (ANN) search → top-K items. 10ms latency for top-200 from 10M items with FAISS.

Why are the towers served independently? Item embeddings are pre-computed and cached in the FAISS index — never recomputed at query time. Only the user embedding is computed online. This is the source of the speed advantage.

Collaborative Filtering (Matrix Factorization)

Decompose the user-item interaction matrix R (shape: users × items) into: R ≈ U × Vᵀ, where U is users × k and V is items × k (k = latent factor dimension)

Training: minimize ||R - UVᵀ||² + λ(||U||² + ||V||²) using ALS or SGD. Inference: candidate score for user i and item j = U_i · V_j

Strengths: captures 'users like you also liked' patterns without explicit item features. Weakness: requires interaction history for both users AND items. Cold start = no embedding.

Content-Based Filtering

Represent items by their content features (category vector, description embedding, tags). Represent user preference as a weighted average of the features of items they engaged with. Score items by cosine similarity between item features and user preference vector.

Strengths: works immediately for new items with features (no interaction history needed). Weakness: over-specialization — a user who clicked 3 mystery novels only gets mystery novels.

Hybrid approach: most production systems blend all three:

• Two-tower for personalized retrieval
• Content-based to cover cold-start items
• Popularity-based as a fallback for cold-start users
• Merge candidates from all sources, deduplicate, pass to ranker