mlsanityData to 14 Oct 2026

Rethinking Chunk Size For Long-Document Retrieval: A Multi-Dataset Analysis

arXiv 2505.21700 ↗First posted 27 May 2025cs.IRcited by 17

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Chunking is a crucial preprocessing step in retrieval-augmented generation (RAG) systems, significantly impacting retrieval effectiveness across diverse datasets. In this study, we systematically evaluate fixed-size chunking strategies and their influence on retrieval performance using multiple embedding models. Our experiments, conducted on both short-form and long-form datasets, reveal that chunk size plays a critical role in retrieval effectiveness -- smaller chunks (64-128 tokens) are optimal for datasets with concise, fact-based answers, whereas larger chunks (512-1024 tokens) improve retrieval in datasets requiring broader contextual understanding. We also analyze the impact of chunking on different embedding models, finding that they exhibit distinct chunking sensitivities. While models like Stella benefit from larger chunks, leveraging global context for long-range retrieval, Snowflake performs better with smaller chunks, excelling at fine-grained, entity-based matching. Our results underscore the trade-offs between chunk size, embedding models, and dataset characteristics, emphasizing the need for improved chunk quality measures, and more comprehensive datasets to advance chunk-based retrieval in long-document Information Retrieval (IR).

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 17 of its 24 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

17 of 24 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  7. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  8. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  10. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  11. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
  12. 2017The NarrativeQA Reading Comprehension Challengecited by 597
5 more
  1. 2016NewsQA: A Machine Comprehension Datasetcited by 331
  2. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  3. 2021Dense Hierarchical Retrieval for Open-Domain Question Answeringcited by 34
  4. 2019The TechQA Datasetcited by 30
  5. 2024Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generationcited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 10.

17 papers.

Sort:

By area:

  1. 2025HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunkingcited by 12
  2. 2025ContextPilot: Fast Long-Context Inference via Context Reusecited by 8
  3. 2025Dynamic Context Adaptation for Consistent Role-Playing Agents with Retrieval-Augmented Generationscited by 4
  4. 2026A Systematic Investigation of Document Chunking Strategies and Embedding Sensitivitycited by 3
  5. 2025An Agile Method for Implementing Retrieval Augmented Generation Tools in Industrial SMEscited by 2
  6. 2025Knowledge-Grounded Agentic Large Language Models for Multi-Hazard Understanding from Reconnaissance Reportscited by 2
  7. 2025Modeling Language as a Sequence of Thoughtscited by 2
  8. 2025SmartChunk Retrieval: Query-Aware Chunk Compression with Planning for Efficient Document RAGcited by 1
  9. 2026The Effect of Text Chunk Size on Retrieval-Augmented Generation Performancecited by 1
  10. 2026RAG Strategies for Natural Language-Based SQL Query and REST API Call Generationcited by 0
  11. 2026QChunker: Learning Question-Aware Text Chunking for Domain RAG via Multi-Agent Debatecited by 0
  12. 2026Domain-Adapted Retrieval for In-Context Annotation of Pedagogical Dialogue Actscited by 0
5 more
  1. 2026MMClima: A Framework for Multimodal Climate Science Data and Evaluationcited by 0
  2. 2026Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregationcited by 0
  3. 2026Cross-Attention Calibrated Deduplication for Retrieval-Augmented Generation Systemcited by 0
  4. 2026Structure-Aware Semantic Chunking with Title-Chain Prefixes: A 1600-Query Evaluation and the Measurement Trap in Text-Transform Ablationscited by 0
  5. 2026Does Document Structure Help Dense Retrieval? A Placebo-Controlled Ablation of Four Mechanisms Across Two Corporacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Chunking: Optimizing Chunking-Method Selection for RAG
  2. 2025Reconstructing Context: Evaluating Advanced Chunking Strategies for Retrieval-Augmented Generation
  3. 2024Is Semantic Chunking Worth the Computational Cost?
  4. 2026Chunking Methods on Retrieval-Augmented Generation - Effectiveness Evaluation Against Computational Cost and Limitations
  5. 2026Reducing Redundancy in Retrieval-Augmented Generation through Chunk Filtering
  6. 2025A New HOPE: Domain-agnostic Automatic Evaluation of Text Chunking
  7. 2026CHOP: Chunkwise Context-Preserving Framework for RAG on Multi Documents
  8. 2025SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension