Contrastive Learning and SBERT Architecture
~45 min read
How SBERT enables efficient sentence comparison with shared-weight siamese encoders, and the contrastive losses used to train it.
Contrastive Learning and SBERT Architecture is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •SBERT uses shared-weight siamese encoders so query and documents can be embedded independently, enabling pre-computation and O(n) retrieval
- •MultipleNegativesRankingLoss requires only (query, positive_doc) pairs — other batch positives become in-batch negatives automatically
- •Larger batch size = more in-batch negatives = stronger training signal for MNRL; 64-256 is a good range