mlsanityData to 14 Oct 2026

RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing

arXiv 2404.19543 ↗First posted 30 Apr 2024cs.CL cs.AIcited by 25

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Large Language Models (LLMs) have catalyzed significant advancements in Natural Language Processing (NLP), yet they encounter challenges such as hallucination and the need for domain-specific knowledge. To mitigate these, recent methodologies have integrated information retrieved from external resources with LLMs, substantially enhancing their performance across NLP tasks. This survey paper addresses the absence of a comprehensive overview on Retrieval-Augmented Language Models (RALMs), both Retrieval-Augmented Generation (RAG) and Retrieval-Augmented Understanding (RAU), providing an in-depth examination of their paradigm, evolution, taxonomy, and applications. The paper discusses the essential components of RALMs, including Retrievers, Language Models, and Augmentations, and how their interactions lead to diverse model structures and applications. RALMs demonstrate utility in a spectrum of tasks, from translation and dialogue systems to knowledge-intensive applications. The survey includes several evaluation methods of RALMs, emphasizing the importance of robustness, accuracy, and relevance in their assessment. It also acknowledges the limitations of RALMs, particularly in retrieval quality and computational efficiency, offering directions for future research. In conclusion, this survey aims to offer a structured insight into RALMs, their potential, and the avenues for their future development in NLP. The paper is supplemented with a Github Repository containing the surveyed works and resources for further study: https://github.com/2471023025/RALM_Survey.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 104 of its 133 references that are in the map. First posted on arXiv 30 Apr 2024.

Builds on

104 of 133 references.

Closest ancestors · 13

Same direction, newest first

  1. 2023InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining11 Oct
  2. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation6 Oct
  3. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context2 Oct
  4. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuning2 Oct
  5. 2023In-Context Retrieval-Augmented Language Models31 Jan
  6. 2023REPLUG: Retrieval-Augmented Black-Box Language Models30 Jan
  7. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP28 Dec
  8. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories20 Dec
  9. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Models5 Aug
  10. 2022Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training21 May
  11. 2021Robust Retrieval Augmented Generation for Zero-shot Slot Filling31 Aug
  12. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
1 more
  1. 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb

Other references · 91

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  10. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  11. 2016Layer Normalizationcited by 5,669
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
38 more
  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  7. 2023Qwen Technical Reportcited by 3,344
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  9. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  10. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  12. 2020GLU Variants Improve Transformercited by 1,804
  13. 2020Knowledge Distillation: A Surveycited by 1,603
  14. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  15. 2019Root Mean Square Layer Normalizationcited by 1,480
  16. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  17. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  18. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  19. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  20. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  21. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  22. 2016Language Modeling with Gated Convolutional Networkscited by 702
  23. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  24. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  25. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  26. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  27. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  28. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 346
  29. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
  30. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 324
  31. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  32. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 276
  33. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 265
  34. 2023Using an LLM to Help With Code Understandingcited by 228
  35. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  36. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 220
  37. 2021Internet-Augmented Dialogue Generationcited by 217
  38. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 210

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 10; 2026: 6.

25 papers.

Sort:

By area:

  1. 2024Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworkscited by 42
  2. 2024Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agentscited by 35
  3. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
  4. 2024Surveying the MLLM Landscape: A Meta-Review of Current Surveyscited by 20
  5. 2024SmartRAG: Jointly Learn RAG-Related Tasks From the Environment Feedbackcited by 19
  6. 2024Towards Reliable Medical Question Answering: Techniques and Challenges in Mitigating Hallucinations in Language Modelscited by 15
  7. 2025AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understandingcited by 15
  8. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
  9. 2025Biomedical Knowledge Graph: A Survey of Domains, Tasks, and Real-World Applicationscited by 6
  10. 2025Affordable AI Assistants with Knowledge Graph of Thoughtscited by 6
  11. 2024A Survey of Retrieval Algorithms in Ad and Content Recommendation Systemscited by 5
  12. 2025DoctorRAG: Medical RAG Fusing Knowledge with Patient Analogy through Textual Gradientscited by 5
13 more
  1. 2025RAGOps: Operating and Managing Retrieval-Augmented Generation Pipelinescited by 5
  2. 2024Retrieval-Augmented Feature Generation for Domain-Specific Classificationcited by 3
  3. 2024A Survey of Query Optimization in Large Language Modelscited by 1
  4. 2025LEKA:LLM-Enhanced Knowledge Augmentationcited by 1
  5. 2025SubGCache: Accelerating Graph-based RAG with Subgraph-level KV Cachecited by 1
  6. 2025ModernBERT + ColBERT: Enhancing biomedical RAG through an advanced re-ranking retrievercited by 1
  7. 2026GraphSeek: Next-Generation Graph Analytics with LLMscited by 1
  8. 2024Thorns and Algorithms: Navigating Generative AI Challenges Inspired by Giraffes and Acaciascited by 0
  9. 2025Smarter, not Bigger: Fine-Tuned RAG-Enhanced LLMs for Automotive HIL Testingcited by 0
  10. 2026PIER: Physics-Informed Environmental Retrieval for Time-Series Modelingcited by 0
  11. 2026When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generationcited by 0
  12. 2026NeuroGraph: An AI Graph-Driven Neuro-Symbolic Framework for Explainable Threat Reasoning in Advanced Manufacturingcited by 0
  13. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Retrieval-Augmented Generation for Natural Language Processing: A Survey
  2. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs
  3. 2024R-Eval: A Unified Toolkit for Evaluating Domain Knowledge of Retrieval Augmented Large Language Models
  4. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
  5. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  6. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
  7. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Models
  8. 2024Evaluation of Retrieval-Augmented Generation: A Survey