Domain Adaptation and Hard Negative Mining
~40 min read
Fine-tuning strategies for domain-specific retrieval: data generation, hard negative mining, and Matryoshka embeddings.
Domain Adaptation and Hard Negative Mining is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Synthetic query generation (LLM writes queries per document) enables no-annotation fine-tuning but requires mixing with real queries to handle short/typo-heavy user queries
- •Hard negatives are semantically close but non-relevant documents — they force the model to learn fine distinctions that random negatives don't teach
- •Matryoshka embeddings (MRL) train the first N dimensions to be maximally informative, enabling dimension truncation at serving time — used by OpenAI text-embedding-3-* models