What is Chunking?
Also called Text Splitting, Document Chunking.
Chunking is the process of splitting documents into smaller passages before they are embedded and indexed. Chunk size and boundary choice determine what a retriever can return, since retrieval operates on whole chunks. Passages that are too large dilute meaning and waste context, while passages that are too small lose the surrounding information needed to interpret them.
A retriever returns chunks, not documents, so chunking decides the unit of evidence. A fifty page handbook indexed as one vector will match almost any question weakly and answer none precisely. The same handbook split by section produces vectors that each represent one topic, which sharpens both the matching and the text placed into the prompt.
Fixed size splitting by character or token count is the simplest approach and the most common starting point, usually with an overlap so a sentence spanning a boundary appears in both neighbors. Structure aware splitting respects headings, list items, table rows, or code blocks, and generally performs better because natural boundaries align with topic boundaries.
Later refinements attach context to each chunk. Prepending the document title and heading path helps a passage stand alone. Storing a small chunk for matching while returning a larger surrounding window for reading separates the retrieval unit from the reading unit, and summarizing a chunk's context into the chunk itself is another documented approach.
No universal optimum exists. The right size depends on document structure, question style, and the reading budget available in the prompt. Teams normally settle it empirically by measuring answer quality on a labeled question set across several configurations, rather than by adopting a default from a tutorial.
Key points
- Splits documents into the passages retrieval will return.
- Overlap prevents ideas from being cut at boundaries.
- Structure aware splits usually beat fixed character counts.
- Titles and headings help a passage stand alone.
- Optimal size is corpus specific and found by testing.
In practice
A twenty page contract is split at clause boundaries, yielding forty chunks that each carry the contract name and clause heading. When someone asks about the notice period for termination, the retriever returns the single termination clause rather than the entire contract, so the model reads three hundred words of exactly relevant text instead of nine thousand words of mostly irrelevant text.