mlsanityData to 14 Oct 2026

When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs

arXiv 2510.09106 ↗First posted 10 Oct 2025cs.CLcited by 1

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) have enabled a wide range of applications through their powerful capabilities in language understanding and generation. However, as LLMs are trained on static corpora, they face difficulties in addressing rapidly evolving information or domain-specific queries. Retrieval-Augmented Generation (RAG) was developed to overcome this limitation by integrating LLMs with external retrieval mechanisms, allowing them to access up-to-date and contextually relevant knowledge. However, as LLMs themselves continue to advance in scale and capability, the relative advantages of traditional RAG frameworks have become less pronounced and necessary. Here, we present a comprehensive review of RAG, beginning with its overarching objectives and core components. We then analyze the key challenges within RAG, highlighting critical weakness that may limit its effectiveness. Finally, we showcase applications where LLMs alone perform inadequately, but where RAG, when combined with LLMs, can substantially enhance their effectiveness. We hope this work will encourage researchers to reconsider the role of RAG and inspire the development of next-generation RAG systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 47 of its 55 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

47 of 55 references.

References · 47

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  10. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  11. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  12. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
35 more
  1. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  2. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  3. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  4. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  5. 2023Active Retrieval Augmented Generationcited by 516
  6. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  7. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  8. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
  9. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
  10. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 281
  11. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  12. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 249
  13. 2025From RAG to Memory: Non-Parametric Continual Learning for Large Language Modelscited by 231
  14. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  15. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certaintycited by 187
  16. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 122
  17. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
  18. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  19. 2024KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generationcited by 67
  20. 2025Retrieval-Augmented Generation with Conflicting Evidencecited by 63
  21. 2025TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Toolscited by 59
  22. 2023Resolving Knowledge Conflicts in Large Language Modelscited by 50
  23. 2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
  24. 2024Gradient based Feature Attribution in Explainable AI: A Technical Reviewcited by 34
  25. 2025Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMscited by 31
  26. 2024To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contextscited by 25
  27. 2024A Survey on Natural Language Counterfactual Generationcited by 21
  28. 2024Mindful-RAG: A Study of Points of Failure in Retrieval Augmented Generationcited by 12
  29. 2025PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chainscited by 10
  30. 2025RoleRAG: Enhancing LLM Role-Playing via Graph Guided Retrievalcited by 7
  31. 2024HyQE: Ranking Contexts with Hypothetical Query Embeddingscited by 6
  32. 2025Measuring Reasoning Utility in LLMs via Conditional Entropy Reductioncited by 6
  33. 2025Response Uncertainty and Probe Modeling: Two Sides of the Same Coin in LLM Interpretability?cited by 3
  34. 2025From Entity-Centric to Goal-Oriented Graphs: Enhancing LLM Knowledge Retrieval in Minecraftcited by 3
  35. 2025Evaluating the Retrieval Robustness of Large Language Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026M-RAG: Semantic Key-Value Indexing for Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Survey
  2. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  3. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  4. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Models
  5. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions
  6. 2024Retrieval-Augmented Generation for Natural Language Processing: A Survey
  7. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  8. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely