Semantic Chunking
~15 min read
Group sentences into a chunk as long as consecutive segments stay semantically similar (via cosine similarity of their embeddings), starting a new chunk exactly where the similarity drops significantly.
Semantic Chunking is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Chunks based on meaning: consecutive segments stay in one chunk while their embeddings remain highly similar
- •A significant drop in cosine similarity between consecutive segments signals a topic shift and starts a new chunk
- •Preserves the natural flow of language and complete ideas — unlike fixed-size chunking
- •Richer, more topically-coherent chunks generally improve retrieval accuracy and downstream response quality
- •The similarity-drop threshold isn't universal — it varies by document type and typically needs per-corpus tuning