Recursive Chunking

Recursive chunking is a two-level approach that tries to get the best of both fixed-size and structure-aware chunking. First, the document is chunked based on inherent, natural separators — paragraph breaks or section markers. Then, each of those chunks is checked against a pre-defined chunk-size limit: if a chunk exceeds that limit, it's split further into smaller pieces; if it already fits within the limit, no further splitting happens.

Concretely: say a document has two paragraphs. Recursive chunking first defines those two paragraphs as two initial chunks. If paragraph 1 is small enough, it stays as-is. If paragraph 2 is too large for the size limit, it gets recursively split into smaller sub-chunks — using the same 'split, then check size, then split again if still too big' logic — until every resulting piece fits the limit.

Like semantic chunking, this approach maintains the natural flow of language and preserves complete ideas wherever possible, because the first-pass split always respects the document's own natural boundaries (paragraphs, sections) rather than an arbitrary character count. The size-limit enforcement only kicks in as a second pass, for chunks that are naturally too large to embed or retrieve efficiently on their own.

The trade-off is implementation and computational overhead: unlike a single fixed-size pass, recursive chunking requires checking sizes and potentially re-splitting at multiple levels, which is more logic to write and more computation to run per document, especially for large corpora with many oversized paragraphs.

def recursive_chunks(text: str, size_limit: int = 500, separators: list[str] = None) -> list[str]:
    """Split on the first available natural separator; recurse into any
    piece still over `size_limit` using the next separator down the list."""
    separators = separators or ["\n\n", "\n", ". ", " "]  # paragraph -> line -> sentence -> word

    if len(text) <= size_limit or not separators:
        return [text]

    sep, rest_seps = separators[0], separators[1:]
    pieces = text.split(sep)

    chunks = []
    for piece in pieces:
        if len(piece) <= size_limit:
            chunks.append(piece)                          # fits — no further splitting
        else:
            chunks.extend(recursive_chunks(piece, size_limit, rest_seps))  # still too big — recurse
    return [c for c in chunks if c.strip()]

doc = "Paragraph one is short.\n\nParagraph two is much longer..."
chunks = recursive_chunks(doc, size_limit=200)