Document Structure-Based Chunking

Document structure-based chunking utilizes the inherent structure of a document — headings, sections, or paragraph markers — to define chunk boundaries directly. Rather than computing anything (no embeddings, no size-based recursion), this strategy simply reads the document's own logical organization and turns each structural unit (e.g., each H2-level section, or each numbered clause) into a chunk.

This approach maintains structural integrity by aligning chunk boundaries exactly with the document's own logical sections — a well-organized manual, contract, or technical spec is already 'pre-chunked' by its authors via headings and sections, and this strategy simply respects that existing organization instead of imposing a different one on top of it. For content that's genuinely structured this way, the resulting chunks tend to align very well with how a human would naturally think about 'one retrievable unit' of that document.

The catch is right there in the description: this approach assumes the document has a clear, consistent structure, which may not actually be true — a scanned PDF, a loosely-formatted blog post, or a document with inconsistent heading usage won't have reliable structural markers to chunk along. And even for well-structured documents, chunks derived this way can vary substantially in length — a short FAQ section next to a sprawling, multi-page technical appendix, both becoming a single 'chunk' — which can mean some chunks exceed an embedding model's token limits. The standard fix is to merge this approach with recursive splitting: use the document's structure for the first-pass boundaries, then apply a size-based recursive split to any resulting section that's still too large.

import re

def document_structure_chunks(markdown_text: str) -> list[dict]:
    """Split a markdown document along its own heading structure —
    each section (from one heading to the next) becomes one chunk."""
    # Split right before any markdown heading (# through ######)
    parts = re.split(r"(?=^#{1,6}\s)", markdown_text, flags=re.MULTILINE)

    chunks = []
    for part in parts:
        part = part.strip()
        if not part:
            continue
        heading_match = re.match(r"^(#{1,6})\s+(.+)", part)
        heading = heading_match.group(2) if heading_match else None
        chunks.append({"heading": heading, "text": part})
    return chunks

# In production: pipe any chunk whose text exceeds your embedding
# model's token limit through recursive_chunks() as a second pass.