# What is Late Chunking?

Late chunking embeds a long document in one pass with a long-context embedding model, then pools the resulting token representations into per-chunk vectors afterward. Because every token was encoded while the whole document was visible, each chunk vector carries context from the rest of the text, unlike conventional chunking where pieces are embedded independently.

Conventional pipelines split first and embed second, so an embedding model sees each chunk in isolation and cannot know what a pronoun refers to or which entity a section discusses. Late chunking reverses the order: the model encodes the full document, producing one representation per token, and chunk boundaries are applied only when those token representations are averaged into fixed length vectors.

The requirement is an embedding model that produces token level outputs and supports a long input window, since the whole document must fit in one pass. The chunk boundaries themselves can still be chosen by any strategy, whether fixed size, sentence based, or structural, so late chunking changes how vectors are computed rather than where the text is cut.

Reported benefits concentrate on documents dense with references across sections, where isolated chunks lose their subject. The technique was introduced by the Jina AI team in 2024 and is comparatively recent, so evidence comes from a limited set of benchmarks and its advantage over simpler alternatives varies by corpus. It should be validated on the actual document collection before adoption.

Operationally it constrains the pipeline. Documents longer than the model's window still need splitting into windows, reintroducing boundary effects at those seams. Re-embedding a single edited chunk means re-encoding the whole document, which raises update cost for frequently changing corpora, and the approach cannot be combined with embedding models that only emit a single pooled vector.

## Key points

- Embeds the whole document first, pools into chunks second
- Chunk vectors retain context from the surrounding text
- Needs a long-context model with token level outputs
- Editing one chunk requires re-encoding the document
- Recent technique, benefits vary by corpus

## In practice

A contract's clause nine reads: this obligation survives termination for three years. Embedded alone, it matches nothing about confidentiality. With late chunking the token representations were produced while the model could see clause eight defining the confidentiality obligation, so the clause nine vector sits near confidentiality survival queries and is retrieved when a lawyer asks how long confidentiality lasts after termination.

## Related terms

- [Chunking](/en/glossary/chunking)
- [Contextual Retrieval](/en/glossary/contextual-retrieval)
- [Embedding](/en/glossary/embedding)
- [Chunk Overlap](/en/glossary/chunk-overlap)
- [Document Ingestion](/en/glossary/document-ingestion)

[Back to the AI Glossary](/en/glossary)