What is Chunk Overlap?
Also called sliding window overlap.
Chunk overlap is the practice of repeating a portion of text at the end of one chunk and the start of the next, so a passage split across a boundary still appears intact somewhere. Overlap is usually expressed as a token or character count, commonly ten to twenty percent of chunk size. It costs storage and duplicate results.
Fixed size splitting cuts wherever the counter runs out, which regularly lands mid-sentence, mid-table, or between a claim and the number that supports it. When that happens neither resulting chunk answers the question well: one has the subject without the value, the other has the value without the subject. Overlap guarantees that a window of text spanning the cut exists as a contiguous span in at least one chunk.
Choosing the amount is a balance. Too little leaves boundary content orphaned; too much inflates the index, since a twenty percent overlap means roughly twenty percent more chunks to store, embed, and search. Overlap also increases the chance that several near-identical chunks occupy the top results, crowding out other sources unless deduplication happens after retrieval.
Overlap is a blunt remedy for a structural problem. Splitting on semantic boundaries such as headings, paragraphs, list items, or sentence groups avoids many bad cuts in the first place, which is why structure-aware splitters usually need less overlap than fixed size ones. For well structured documents, zero or minimal overlap with clean boundaries often outperforms heavy overlap with arbitrary ones.
Downstream handling matters as much as the setting itself. Retrieved chunks that overlap should be merged before being placed in a prompt, so the model does not see the same sentences twice and waste context. Systems that track parent document offsets can reconstruct a clean contiguous span from several overlapping hits, which gives the benefit of overlap without the duplication in the final context.
Key points
- Repeats text across boundaries so split passages stay intact
- Commonly ten to twenty percent of chunk size
- Inflates index size and can crowd results with near-duplicates
- Structure-aware splitting reduces the need for overlap
- Overlapping hits should be merged before prompting
In practice
A refund policy sentence spans a chunk boundary: the first chunk ends with refunds are issued within, the second begins with fourteen business days of approval. With no overlap, neither chunk answers how long a refund takes. With a fifty-token overlap, the second chunk starts early enough to contain the full sentence, and the query about refund timing retrieves a passage that actually states the number.