mlsanityData to 7 Oct 2026

The Power of Noise: Redefining Retrieval for RAG Systems

arXiv 2401.14887 ↗First posted 26 Jan 2024cs.IR cs.CLcited by 275

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has recently emerged as a method to extend beyond the pre-trained knowledge of Large Language Models by augmenting the original prompt with relevant passages or documents retrieved by an Information Retrieval (IR) system. RAG has become increasingly important for Generative AI solutions, especially in enterprise settings or in any domain in which knowledge is constantly refreshed and cannot be memorized in the LLM. We argue here that the retrieval component of RAG systems, be it dense or sparse, deserves increased attention from the research community, and accordingly, we conduct the first comprehensive and systematic examination of the retrieval strategy of RAG systems. We focus, in particular, on the type of passages IR systems within a RAG solution should retrieve. Our analysis considers multiple factors, such as the relevance of the passages included in the prompt context, their position, and their number. One counter-intuitive finding of this work is that the retriever's highest-scoring documents that are not directly relevant to the query (e.g., do not contain the answer) negatively impact the effectiveness of the LLM. Even more surprising, we discovered that adding random documents in the prompt improves the LLM accuracy by up to 35%. These results highlight the need to investigate the appropriate strategies when integrating retrieval with LLMs, thereby laying the groundwork for future research in this area.

Key facts. Cited by 275 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 35 of its 59 references that are in the map. First posted on arXiv 26 Jan 2024.

Builds on

35 of 59 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Bridging the Preference Gap between Retrievers and LLMs13 Jan

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  10. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  11. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  12. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
22 more
  1. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  2. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  3. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  4. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  5. 2020Recipes for building an open-domain chatbotcited by 593
  6. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  7. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  8. 2023Augmented Language Models: a Surveycited by 398
  9. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  10. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  11. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  12. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 154
  13. 2018Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Contextcited by 113
  14. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  15. 2022Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Listscited by 54
  16. 2021UnitedQA: A Hybrid Approach for Open Domain Question Answeringcited by 42
  17. 2022Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Modelcited by 32
  18. 2023ChatGPT Hallucinates when Attributing Answerscited by 14
  19. 2023Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problemscited by 14
  20. 2023Multimodal Neural Databasescited by 7
  21. 2023RRAML: Reinforced Retrieval Augmented Machine Learningcited by 6
  22. 2023Prompt-to-OS (P2OS): Revolutionizing Operating Systems and Human-Computer Interaction with Integrated AI Generative Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 71; 2025: 107; 2026: 97.

275 papers.

Sort:

By area:

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  2. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  3. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  4. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 170
  5. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  6. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAGcited by 122
  7. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 118
  8. 2026MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memorycited by 113
  9. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  10. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  11. 2025MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilotcited by 83
  12. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
38 more
  1. 2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systemscited by 69
  2. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbationscited by 64
  3. 2025PathRAG: Pruning Graph-based Retrieval Augmented Generation with Relational Pathscited by 60
  4. 2025KVLink: Accelerating Large Language Models via Efficient KV Cache Reusecited by 60
  5. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 57
  6. 2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledgecited by 49
  7. 2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
  8. 2024Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworkscited by 41
  9. 2025Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learningcited by 39
  10. 2025Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generationcited by 36
  11. 2024Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?cited by 34
  12. 2025The Distracting Effect: Understanding Irrelevant Passages in RAGcited by 32
  13. 2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generationcited by 31
  14. 2024How Much Can RAG Help the Reasoning of LLM?cited by 31
  15. 2024Beyond Text: Optimizing RAG with Multimodal Inputs for Industrial Applicationscited by 31
  16. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  17. 2024REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answeringcited by 28
  18. 2024Fine Tuning LLM for Enterprise: Practical Guidelines and Recommendationscited by 28
  19. 2024Customized Retrieval Augmented Generation and Benchmarking for EDA Tool Documentation QAcited by 28
  20. 2024TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systemscited by 28
  21. 2025Traceback of Poisoning Attacks to Retrieval-Augmented Generationcited by 28
  22. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievalscited by 27
  23. 2024Context Embeddings for Efficient Answer Generation in RAGcited by 26
  24. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontierscited by 26
  25. 2024RAGGED: Towards Informed Design of Scalable and Stable RAG Systemscited by 24
  26. 2024HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMscited by 24
  27. 2024T-RAG: Lessons from the LLM Trenchescited by 23
  28. 2024A Survey of Conversational Searchcited by 22
  29. 2025SlideCoder: Layout-aware RAG-enhanced Hierarchical Slide Generation from Designcited by 22
  30. 2024AmbigDocs: Reasoning across Documents on Different Entities under the Same Namecited by 21
  31. 2024Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspectivecited by 20
  32. 2025U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystackcited by 20
  33. 2024FeB4RAG: Evaluating Federated Search in the Context of Retrieval Augmented Generationcited by 19
  34. 2024ATM: Adversarial Tuning Multi-agent System Makes a Robust Retrieval-Augmented Generatorcited by 19
  35. 2024An overview of domain-specific foundation model: key technologies, applications and challengescited by 19
  36. 2025SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidencecited by 18
  37. 2024LLMs Know What They Need: Leveraging a Missing Information Guided Framework to Empower Retrieval-Augmented Generationcited by 17
  38. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 17

50 most cited of 275 All 275 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
  2. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  3. 2024Developing Retrieval Augmented Generation (RAG) based LLM Systems from PDFs: An Experience Report
  4. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  5. 2026Is Agentic RAG worth it? An experimental comparison of RAG approaches
  6. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generation
  7. 2025Enhancing Retrieval-Augmented Generation: A Study of Best Practices
  8. 2024Toward Optimal Search and Retrieval for RAG