mlsanityData to 14 Oct 2026

More Documents, Same Length: Isolating the Challenge of Multiple Documents in RAG

arXiv 2503.04388 ↗First posted 6 Mar 2025cs.CLcited by 16

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) enhances the accuracy of Large Language Model (LLM) responses by leveraging relevant external documents during generation. Although previous studies noted that retrieving many documents can degrade performance, they did not isolate how the quantity of documents affects performance while controlling for context length. We evaluate various language models on custom datasets derived from a multi-hop QA task. We keep the context length and position of relevant information constant while varying the number of documents, and find that increasing the document count in RAG settings poses significant challenges for most LLMs, reducing performance by up to 20%. However, Qwen2.5 maintained consistent results across increasing document counts, indicating better multi-document handling capability. Finally, our results indicate that processing multiple documents is a separate challenge from handling long contexts. We also make the datasets and code available: https://github.com/shaharl6000/MoreDocsSameLen .

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 12 of its 17 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

12 of 17 references.

References · 12

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2024Qwen2 Technical Reportcited by 2,120
  5. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  6. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  7. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  8. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 122
  9. 2024Steering Large Language Models between Code Execution and Textual Reasoningcited by 24
  10. 2024SEAM: A Stochastic Benchmark for Multi-Document Taskscited by 8
  11. 2023Revisiting Sentence Union Generation as a Testbed for Text Consolidationcited by 6
  12. 2024TeleOracle: Fine-Tuned Retrieval-Augmented Generation with Long-Context Support for Networkcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 10.

16 papers.

Sort:

By area:

  1. 2025Hierarchical Document Refinement for Long-context Retrieval-augmented Generationcited by 16
  2. 2025Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Findcited by 7
  3. 2026Agent Skills Can Be Harmful: An Empirical Study of Skill-Induced Failures in LLM Agentscited by 6
  4. 2026TASR: Training-Free Adaptive Stopping for Iterative Retrievalcited by 5
  5. 2025Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Boundscited by 4
  6. 2025RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generationcited by 3
  7. 2026Failure Modes in Multi-Hop QA: The Weakest Link Effect and the Recognition Bottleneckcited by 3
  8. 2026RAGExplorer: A Visual Analytics System for the Comparative Diagnosis of RAG Systemscited by 2
  9. 2026Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Lengthcited by 2
  10. 2026Are Finer Citations Always Better? Rethinking Granularity for Attributed Generationcited by 2
  11. 2026Navigating Large-Scale Document Collections: MuDABench for Multi-Document Analytical QAcited by 2
  12. 2025MeVe: A Modular System for Memory Verification and Effective Context Control in Language Modelscited by 1
4 more
  1. 2026IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answeringcited by 1
  2. 2025Is Implicit Knowledge Enough for LLMs? A RAG Approach for Tree-based Structurescited by 0
  3. 2026The Recall Trap: A Recall-Maximizing Retriever Configuration Reduces Issue Resolution in Fixed-Budget Code Contextcited by 0
  4. 2026MegaMem: A Retrieval Solution for Ultra-Large Context Windowscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Long Context RAG Performance of Large Language Models
  2. 2025Less LLM, More Documents: Searching for Improved RAG
  3. 2025Stronger Baselines for Retrieval-Augmented Generation with Long-Context Language Models
  4. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  5. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
  6. 2025Enhancing Retrieval-Augmented Generation: A Study of Best Practices
  7. 2024Optimizing Query Generation for Enhanced Document Retrieval in RAG
  8. 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?