mlsanityData to 14 Oct 2026

Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering

arXiv 2210.02627 ↗First posted 6 Oct 2022cs.CL cs.AI cs.IRcited by 127

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval Augment Generation (RAG) is a recent advancement in Open-Domain Question Answering (ODQA). RAG has only been trained and explored with a Wikipedia-based external knowledge base and is not optimized for use in other specialized domains such as healthcare and news. In this paper, we evaluate the impact of joint training of the retriever and generator components of RAG for the task of domain adaptation in ODQA. We propose textitRAG-end2end, an extension to RAG, that can adapt to a domain-specific knowledge base by updating all components of the external knowledge base during training. In addition, we introduce an auxiliary training signal to inject more domain-specific knowledge. This auxiliary signal forces textitRAG-end2end to reconstruct a given sentence by accessing the relevant information from the external knowledge base. Our novel contribution is unlike RAG, RAG-end2end does joint training of the retriever and generator for the end QA task and domain adaptation. We evaluate our approach with datasets from three domains: COVID-19, News, and Conversations, and achieve significant performance improvements compared to the original RAG model. Our work has been open-sourced through the Huggingface Transformers library, attesting to our work's credibility and technical consistency.

Key facts. Cited by 127 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 34 of its 45 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

34 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering9 Jun

Other references · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  9. 2017Billion-scale similarity search with GPUscited by 2,497
  10. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  12. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
21 more
  1. 2015Teaching Machines to Read and Comprehendcited by 1,049
  2. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  3. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  4. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  5. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 632
  6. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  7. 2016NewsQA: A Machine Comprehension Datasetcited by 331
  8. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 267
  9. 2021Internet-Augmented Dialogue Generationcited by 217
  10. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  11. 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
  12. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  13. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 138
  14. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 129
  15. 2020Pre-training via Paraphrasingcited by 92
  16. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  17. 2020End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systemscited by 65
  18. 2021NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learnedcited by 52
  19. 2021Controllable Abstractive Dialogue Summarization with Sketch Supervisioncited by 27
  20. 2021QAConv: Question Answering on Informative Conversationscited by 17
  21. 2020Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 10; 2024: 50; 2025: 54; 2026: 12.

127 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  2. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  3. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  4. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 176
  5. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
  6. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 143
  7. 2024Seven Failure Points When Engineering a Retrieval Augmented Generation Systemcited by 139
  8. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 117
  9. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
  10. 2024RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generationcited by 98
  11. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Modelscited by 86
  12. 2024Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrieverscited by 84
38 more
  1. 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generationcited by 75
  2. 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Textcited by 63
  3. 2024Don't Forget to Connect! Improving RAG with Graph-based Rerankingcited by 49
  4. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 46
  5. 2025In-depth Analysis of Graph-based RAG in a Unified Frameworkcited by 46
  6. 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
  7. 2023Chainpoll: A high efficacy method for LLM hallucination detectioncited by 39
  8. 2023KnowGPT: Knowledge Graph based Prompting for Large Language Modelscited by 31
  9. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 31
  10. 2024PersonaRAG: Enhancing Retrieval-Augmented Generation Systems with User-Centric Agentscited by 30
  11. 2025SGMem: Sentence Graph Memory for Long-Term Conversational Agentscited by 29
  12. 2025A Survey of Personalization: From RAG to Agentcited by 27
  13. 2024Telco-RAG: Navigating the Challenges of Retrieval-Augmented Language Models for Telecommunicationscited by 26
  14. 2023Chemist-X: Large Language Model-empowered Agent for Reaction Condition Recommendation in Chemical Synthesiscited by 25
  15. 2024When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?cited by 24
  16. 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answeringcited by 23
  17. 2024Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Modelscited by 23
  18. 2024ConfusedPilot: Confused Deputy Risks in RAG-based LLMscited by 22
  19. 2024Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Modelscited by 21
  20. 2025Benchmarking Retrieval-Augmented Generation for Chemistrycited by 20
  21. 2025RANGE: Retrieval Augmented Neural Fields for Multi-Resolution Geo-Embeddingscited by 19
  22. 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 18
  23. 2025Fine-tuning a Large Language Model for Automating Computational Fluid Dynamics Simulationscited by 18
  24. 2023Improving Retrieval-Augmented Large Language Models via Data Importance Learningcited by 17
  25. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queriescited by 16
  26. 2025Joint-GCG: Unified Gradient-Based Poisoning Attacks on Retrieval-Augmented Generation Systemscited by 16
  27. 2025BookRAG: A Hierarchical Structure-aware Index-based Approach for Retrieval-Augmented Generation on Complex Documentscited by 16
  28. 2024Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximizationcited by 14
  29. 2024Accelerating Retrieval-Augmented Generationcited by 14
  30. 2024Towards LifeSpan Cognitive Systemscited by 13
  31. 2025Spatial-RAG: Spatial Retrieval Augmented Generation for Real-World Geospatial Reasoning Questionscited by 13
  32. 2026Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-offcited by 13
  33. 2024Mindful-RAG: A Study of Points of Failure in Retrieval Augmented Generationcited by 12
  34. 2024Automatic Database Configuration Debugging using Retrieval-Augmented Language Modelscited by 12
  35. 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answeringcited by 12
  36. 2025ArgRAG: Explainable Retrieval Augmented Generation using Quantitative Bipolar Argumentationcited by 12
  37. 2023Evaluating ChatGPT as a Question Answering System: A Comprehensive Analysis and Comparison with Existing Modelscited by 11
  38. 2023X-TRA: Improving Chest X-ray Tasks with Cross-Modal Retrieval Augmentationcited by 10

50 most cited of 127 All 127 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation
  2. 2024W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering
  3. 2024Retrieval-Augmented Generation for Domain-Specific Question Answering: A Case Study on Pittsburgh and CMU
  4. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains
  5. 2025Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generation
  6. 2025Domain-Specific Data Generation Framework for RAG Adaptation
  7. 2024REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering
  8. 2025QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance