# Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models

arXiv: https://arxiv.org/abs/2409.04701 · HTML: https://mlsanity.com/paper/2409.04701/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 7 Sep 2024.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents](https://mlsanity.com/paper/2310.19923/) (2023)
- [Dense X Retrieval: What Retrieval Granularity Should We Use?](https://mlsanity.com/paper/2312.06648/) (2023)
- [LongEmbed: Extending Embedding Models for Long Context Retrieval](https://mlsanity.com/paper/2404.12096/) (2024)
- [Jina-ColBERT-v2: A General-Purpose Multilingual Late Interaction Retriever](https://mlsanity.com/paper/2408.16672/) (2024)

## Built on by (most cited first, 30 of 73)

- [A Survey on Knowledge-Oriented Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10677/) (2025)
- [jina-reranker-v3: Last but Not Late Interaction for Listwise Document Reranking](https://mlsanity.com/paper/2509.25085/) (2025)
- [U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack](https://mlsanity.com/paper/2503.00353/) (2025)
- [MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings](https://mlsanity.com/paper/2506.23115/) (2025)
- [EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation](https://mlsanity.com/paper/2412.12559/) (2024)
- [Should We Still Pretrain Encoders with Masked Language Modeling?](https://mlsanity.com/paper/2507.00994/) (2025)
- [Reconstructing Context: Evaluating Advanced Chunking Strategies for Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.19754/) (2025)
- [HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking](https://mlsanity.com/paper/2509.11552/) (2025)
- [Towards Reliable Retrieval in RAG Systems for Large Legal Datasets](https://mlsanity.com/paper/2510.06999/) (2025)
- [VTCBench: Can Vision-Language Models Understand Long Context with Vision-Text Compression?](https://mlsanity.com/paper/2512.15649/) (2025)
- [MultiFinRAG: An Optimized Multimodal Retrieval-Augmented Generation (RAG) Framework for Financial Question Answering](https://mlsanity.com/paper/2506.20821/) (2025)
- [Diffusion-Pretrained Dense and Contextual Embeddings](https://mlsanity.com/paper/2602.11151/) (2026)
- [Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval](https://mlsanity.com/paper/2604.10167/) (2026)
- [HeteRAG: A Heterogeneous Retrieval-augmented Generation Framework with Decoupled Knowledge Representations](https://mlsanity.com/paper/2504.10529/) (2025)
- [DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression](https://mlsanity.com/paper/2603.15685/) (2026)
- [LLM as HPC Expert: Extending RAG Architecture for HPC Data](https://mlsanity.com/paper/2501.14733/) (2024)
- [Total Recall QA: A Verifiable Evaluation Suite for Deep Research Agents](https://mlsanity.com/paper/2603.18516/) (2026)
- [FinDebate: Multi-Agent Collaborative Intelligence for Financial Analysis](https://mlsanity.com/paper/2509.17395/) (2025)
- [AI for Climate Finance: Agentic Retrieval and Multi-Step Reasoning for Early Warning System Investments](https://mlsanity.com/paper/2504.05104/) (2025)
- [CoRet: Improved Retriever for Code Editing](https://mlsanity.com/paper/2505.24715/) (2025)
- [Contrastive Learning with Narrative Twins for Modeling Story Salience](https://mlsanity.com/paper/2601.07765/) (2026)
- [Information Representation Fairness in Long-Document Embeddings: The Peculiar Interaction of Positional and Language Bias](https://mlsanity.com/paper/2601.16934/) (2026)
- [From BM25 to Corrective RAG: Benchmarking Retrieval Strategies for Text-and-Table Documents](https://mlsanity.com/paper/2604.01733/) (2026)
- [SpeakEasy: Enhancing Text-to-Speech Interactions for Expressive Content Creation](https://mlsanity.com/paper/2504.05106/) (2025)
- [OmniLLP: Enhancing LLM-based Log Level Prediction with Context-Aware Retrieval](https://mlsanity.com/paper/2508.08545/) (2025)
- [Cross-Document Topic-Aligned Chunking for Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.05265/) (2025)
- [SmartChunk Retrieval: Query-Aware Chunk Compression with Planning for Efficient Document RAG](https://mlsanity.com/paper/2602.22225/) (2025)
- [KohakuRAG: A simple RAG framework with hierarchical document indexing](https://mlsanity.com/paper/2603.07612/) (2026)
- [Graph-Aware Late Chunking for Retrieval-Augmented Generation in Biomedical Literature](https://mlsanity.com/paper/2603.22633/) (2026)
- [Adaptive Chunking: Optimizing Chunking-Method Selection for RAG](https://mlsanity.com/paper/2603.25333/) (2026)

## Abstract (arXiv)

Many use cases require retrieving smaller portions of text, and dense vector-based retrieval systems often perform better with shorter text segments, as the semantics are less likely to be over-compressed in the embeddings. Consequently, practitioners often split text documents into smaller chunks and encode them separately. However, chunk embeddings created in this way can lose contextual information from surrounding chunks, resulting in sub-optimal representations. In this paper, we introduce a novel method called late chunking, which leverages long context embedding models to first embed all tokens of the long text, with chunking applied after the transformer model and just before mean pooling - hence the term late in its naming. The resulting chunk embeddings capture the full contextual information, leading to superior results across various retrieval tasks. The method is generic enough to be applied to a wide range of long-context embedding models and works without additional training. To further increase the effectiveness of late chunking, we propose a dedicated fine-tuning approach for embedding models.