mlsanityData to 14 Oct 2026

A Systematic Analysis of Chunking Strategies for Reliable Question Answering

arXiv 2601.14123 ↗First posted 20 Jan 2026cs.CL cs.IRcited by 0

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

We study how document chunking choices impact the reliability of Retrieval-Augmented Generation (RAG) systems in industry. While practice often relies on heuristics, our end-to-end evaluation on Natural Questions systematically varies chunking method (token, sentence, semantic, code), chunk size, overlap, and context length. We use a standard industrial setup: SPLADE retrieval and a Mistral-8B generator. We derive actionable lessons for cost-efficient deployment: (i) overlap provides no measurable benefit and increases indexing cost; (ii) sentence chunking is the most cost-effective method, matching semantic chunking up to ~5k tokens; (iii) a "context cliff" reduces quality beyond ~2.5k tokens; and (iv) optimal context depends on the goal (semantic quality peaks at small contexts; exact match at larger ones).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 5 of its 9 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

5 of 9 references.

References · 5

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  3. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
  4. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 195
  5. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 127

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluating Chunking Strategies for Retrieval-Augmented Generation on Academic Texts
  2. 2024Is Semantic Chunking Worth the Computational Cost?
  3. 2025A New HOPE: Domain-agnostic Automatic Evaluation of Text Chunking
  4. 2026Comparing Chunking and Embedding Strategies for Turkish RAG Systems
  5. 2026Query-Adaptive Semantic Chunking for Retrieval-Augmented Generation: A Dynamic Strategy with Contextual Window Expansion
  6. 2026Structure-Aware Semantic Chunking with Title-Chain Prefixes: A 1600-Query Evaluation and the Measurement Trap in Text-Transform Ablations
  7. 2025Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter Impact on Performance and Efficiency
  8. 2026A Systems-Level Analysis of Sensitivity, Robustness, and Stability in Retrieval-Augmented Generation