🤖

BERT — Bidirectional Encoders

BERT configurations, pre-training, fine-tuning, contextual embeddings

What is BERT — Bidirectional Encoder Representations

BERT definition, why bidirectional matters vs GPT unidirectional

Layer counts, hidden dims, attention heads, parameter counts for each

Three embedding types summed together, [CLS] and [SEP] tokens

15% token masking, predict the missing word, bidirectional context advantage

Sentence A + Sentence B, IsNext vs NotNext classification

Classification, NER, QA — how to add task-specific heads, training strategy

Same word different contexts: bank in finance vs river, BERT vs Word2Vec

Improvements over BERT, efficiency tradeoffs, which to use when