mlsanityData to 7 Oct 2026

Retrieval-Augmented Generation for Natural Language Processing: A Survey

arXiv 2407.13193 ↗First posted 18 Jul 2024cs.CLcited by 70

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Large language models (LLMs) have achieved strong empirical performance in various fields, benefiting from their huge amount of parameters that store knowledge. However, LLMs still suffer from several key issues, such as hallucination problems, knowledge update issues, and lacking domain-specific expertise. The appearance of retrieval-augmented generation (RAG), which leverages an external knowledge base to augment LLMs, mitigates these limitations. This paper presents a systematic review of RAG techniques for natural language processing (NLP), with a focus on retrievers and retrieval fusions. We introduce a novel taxonomy of retrieval fusions, such as query-based, logits-based, latent, and parametric fusion, and provide structured comparisons across accessibility, efficiency, and use cases. The paper further examines RAG applications across diverse NLP tasks, discusses evaluation methodologies and benchmark limitations, and analyzes training paradigms with and without knowledge base updates. Finally, we explore industrial deployment considerations and identify emerging challenges and future directions, including security, efficiency, and graph-based retrieval.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 98 of its 163 references that are in the map. First posted on arXiv 18 Jul 2024.

Builds on

98 of 163 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems25 Jun
  2. 2024UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysis21 Jun
  3. 2024PipeRAG: Fast Retrieval-Augmented Generation via Algorithm-System Co-design8 Mar
  4. 2024Retrieval-Augmented Generation for AI-Generated Content: A Survey29 Feb
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec
  6. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems16 Nov
  7. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory3 May
  8. 2022FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation28 Sep
  9. 2022A Survey on Retrieval-Augmented Text Generation2 Feb

Other references · 89

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  9. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  11. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
38 more
  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2017Billion-scale similarity search with GPUscited by 2,477
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  10. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  11. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  12. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  13. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  14. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  15. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  16. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  17. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  18. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  19. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  20. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  21. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  22. 2024Understanding the planning of LLM agents: A surveycited by 414
  23. 2023Adapting Language Models to Compress Contextscited by 350
  24. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  25. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  26. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  27. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  28. 2020Nearest Neighbor Machine Translationcited by 209
  29. 2018Multimodal Generative Models for Scalable Weakly-Supervised Learningcited by 200
  30. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  31. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  32. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 175
  33. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  34. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  35. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 136
  36. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  37. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  38. 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 127

50 most cited of 89 All 89 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 41; 2026: 22.

70 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  2. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  3. 2025In-depth Analysis of Graph-based RAG in a Unified Frameworkcited by 45
  4. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 34
  5. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Modelscited by 32
  6. 2025ArchRAG: Attributed Community-based Hierarchical Retrieval-Augmented Generationcited by 27
  7. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  8. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  9. 2025Enabling Personalized Long-term Interactions in LLM-based Agents through Persistent Memory and User Profilescited by 16
  10. 2025InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented Generationcited by 15
  11. 2025EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corporacited by 14
  12. 2025Retrv-R1: A Reasoning-Driven MLLM Framework for Universal and Efficient Multimodal Retrievalcited by 12
38 more
  1. 2025SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAGcited by 12
  2. 2025MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystackscited by 11
  3. 2025CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoningcited by 9
  4. 2024A Hybrid RAG System with Comprehensive Enhancement on Complex Reasoningcited by 8
  5. 2025VerilogDB: The Largest, Highest-Quality Dataset with a Preprocessing Framework for LLM-based RTL Generationcited by 8
  6. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  7. 2025AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoningcited by 7
  8. 2025AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMscited by 7
  9. 2025Talk to Right Specialists: Iterative Routing in Multi-agent Systems for Question Answeringcited by 6
  10. 2025Agentic LLM Framework for Adaptive Decision Discoursecited by 6
  11. 2025ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Rerankingcited by 6
  12. 2025A Survey on Proactive Defense Strategies Against Misinformation in Large Language Modelscited by 5
  13. 2025RAG for Geoscience: What We Expect, Gaps and Opportunitiescited by 5
  14. 2025Retrieval as a Decision: Training-Free Adaptive Gating for Efficient RAGcited by 5
  15. 2026Optimizing User Profiles via Contextual Bandits for Retrieval-Augmented LLM Personalizationcited by 5
  16. 2024Self-adaptive Multimodal Retrieval-Augmented Generationcited by 4
  17. 2025Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoningcited by 4
  18. 2025Clue-RAG: Towards Accurate and Cost-Efficient Graph-based RAG via Multi-Partite Graph and Query-Driven Iterative Retrievalcited by 4
  19. 2025Adversarial Generation and Collaborative Evolution of Safety-Critical Scenarios for Autonomous Vehiclescited by 4
  20. 2026Memory Bank Compression for Continual Adaptation of Large Language Modelscited by 4
  21. 2026Efficient Privacy-Preserving Retrieval Augmented Generation with Distance-Preserving Encryptioncited by 4
  22. 2025Bye-bye, Bluebook? Automating Legal Drudgery With AI-Augmented Rule Followingcited by 3
  23. 2026Multimodal Adaptive Retrieval Augmented Generation through Internal Representation Learningcited by 3
  24. 2026From Soliloquy to Agora: Memory-Enhanced LLM Agents with Decentralized Debate for Optimization Modelingcited by 3
  25. 2024LLM-POTUS Score: A Framework of Analyzing Presidential Debates with Large Language Modelscited by 2
  26. 2024TOBUGraph: Knowledge Graph-Based Retrieval for Enhanced LLM Performance Beyond RAGcited by 2
  27. 2025QuOTE: Question-Oriented Text Embeddingscited by 2
  28. 2025Evaluating the Retrieval Robustness of Large Language Modelscited by 2
  29. 2025Hardness-Aware Dynamic Curriculum Learning for Robust Multimodal Emotion Recognition with Missing Modalitiescited by 2
  30. 2025ReGeS: Reciprocal Retrieval-Generation Synergy for Conversational Recommender Systemscited by 2
  31. 2025Retrieval-Augmented Multi-LLM Ensemble for Industrial Part Specification Extractioncited by 2
  32. 2026Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directionscited by 2
  33. 2026IntrAgent: An LLM Agent for Content-Grounded Information Retrieval through Literature Reviewcited by 2
  34. 2024Deploying Large Language Models With Retrieval Augmented Generationcited by 1
  35. 2024A Survey of Query Optimization in Large Language Modelscited by 1
  36. 2025Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answeringcited by 1
  37. 2025Optimization of embeddings storage for RAG systems using quantization and dimensionality reduction techniquescited by 1
  38. 2025Human-Centered Evaluation of RAG outputs: a framework and questionnaire for human-AI collaborationcited by 1

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  2. 2024Evaluation of Retrieval-Augmented Generation: A Survey
  3. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions
  4. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  5. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs
  6. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Models
  7. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing
  8. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers