mlsanityData to 7 Oct 2026

Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generation

arXiv 2504.02411 ↗First posted 3 Apr 2025cs.CLcited by 3

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) enhances LLM factuality, but multi-domain applications face challenges like lack of diverse benchmarks and poor out-of-domain generalization. The first contribution of this work is to introduce a diverse benchmark comprising a variety of question-answering tasks from 8 sources and covering 13 domains. Our second contribution consists in systematically testing out-of-domain generalization for typical RAG tuning strategies. While our findings reveal that standard fine-tuning fails to generalize effectively, we show that sequence-level distillation with teacher-generated labels improves out-of-domain performance by providing more coherent supervision. Our findings highlight key strategies for improving multi-domain RAG robustness.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 32 of its 43 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

32 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering19 Jul

Other references · 31

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  9. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  10. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  11. 2016Sequence-Level Knowledge Distillationcited by 761
  12. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
19 more
  1. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  2. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  3. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  4. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  5. 2020CORD-19: The COVID-19 Open Research Datasetcited by 179
  6. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  7. 2024Long-form factuality in large language modelscited by 146
  8. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  9. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 121
  10. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  11. 2018DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehensioncited by 70
  12. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  13. 2020Evaluating Robustness to Input Perturbations for Neural Machine Translationcited by 36
  14. 2024Examining the robustness of LLM evaluation to the distributional assumptions of benchmarkscited by 31
  15. 2019The TechQA Datasetcited by 29
  16. 2023A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepediacited by 24
  17. 2025PISCO: Pretty Simple Compression for Retrieval-Augmented Generationcited by 20
  18. 2024Stronger Models are NOT Stronger Teachers for Instruction Tuningcited by 16
  19. 2023Measuring the Robustness of NLP Models to Domain Shiftscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surfacecited by 8
  2. 2025PowerChain: A Verifiable Agentic AI System for Automating Distribution Grid Analysescited by 6
  3. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains
  2. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  3. 2024Retrieval-Augmented Generation for Domain-Specific Question Answering: A Case Study on Pittsburgh and CMU
  4. 2025Domain-Specific Data Generation Framework for RAG Adaptation
  5. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  6. 2024W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering
  7. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions
  8. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions