mlsanityData to 7 Oct 2026

SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains

arXiv 2410.17952 ↗First posted 23 Oct 2024cs.CL cs.AI cs.IRcited by 28

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

also in 1 more
Abstract (arXiv)

Retrieval-augmented generation (RAG) enhances the question-answering (QA) abilities of large language models (LLMs) by integrating external knowledge. However, adapting general-purpose RAG systems to specialized fields such as science and medicine poses unique challenges due to distribution shifts and limited access to domain-specific data. To tackle this, we propose SimRAG, a self-training approach that equips the LLM with joint capabilities of question answering and question generation for domain adaptation. Our method first fine-tunes the LLM on instruction-following, question-answering, and search-related data. Then, it prompts the same LLM to generate diverse domain-relevant questions from unlabeled corpora, with an additional filtering strategy to retain high-quality synthetic examples. By leveraging these self-generated synthetic examples, the LLM can improve their performance on domain-specific RAG tasks. Experiments on 11 datasets, spanning two backbone sizes and three domains, demonstrate that SimRAG outperforms baselines by 1.2%--8.6%.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 59 of its 88 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

59 of 88 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity21 Mar
  2. 2024BlendFilter: Advancing Retrieval-Augmented Large Language Models via Query Generation Blending and Knowledge Filtering16 Feb
  3. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAG18 Jan
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 55

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  8. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  9. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  10. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  11. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  12. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
38 more
  1. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  2. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  3. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  4. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  5. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  6. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  7. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  8. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  9. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  10. 2019ELI5: Long Form Question Answeringcited by 567
  11. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 507
  12. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  13. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  14. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  15. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  16. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  17. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  18. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  19. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  20. 2024Iterative Reasoning Preference Optimizationcited by 244
  21. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  22. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  23. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  24. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  25. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  26. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 166
  27. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  28. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  29. 2020The Effect of Natural Distribution Shift on Question Answering Modelscited by 104
  30. 2019Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoningcited by 100
  31. 2023WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferencescited by 100
  32. 2020Self-training Improves Pre-training for Natural Language Understandingcited by 96
  33. 2023AlpaCare:Instruction-tuned Large Language Models for Medical Applicationcited by 88
  34. 2019Reasoning Over Paragraph Effects in Situationscited by 79
  35. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  36. 2024Instruction Pre-Training: Language Models are Supervised Multitask Learnerscited by 63
  37. 2023InstructRetro: Instruction Tuning post Retrieval-Augmented Pretrainingcited by 53
  38. 2024BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrieverscited by 38

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 15; 2026: 11.

28 papers.

Sort:

By area:

  1. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontierscited by 26
  2. 2025Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaborationcited by 19
  3. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 17
  4. 2026Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gaincited by 16
  5. 2025Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillationcited by 15
  6. 2025Retrieval-augmented GUI Agents with Generative Guidelinescited by 14
  7. 2025RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimizationcited by 9
  8. 2025SearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering?cited by 9
  9. 2025Your RAG is Unfair: Exposing Fairness Vulnerabilities in Retrieval-Augmented Generation via Backdoor Attackscited by 8
  10. 2025Optimizing Case-Based Reasoning System for Functional Test Script Generation with Large Language Modelscited by 7
  11. 2025Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editingcited by 6
  12. 2025DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillationcited by 5
16 more
  1. 2024A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detectioncited by 3
  2. 2025A Survey on Foundation Models for Personalized Federated Intelligencecited by 2
  3. 2025RAR2: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrievalcited by 2
  4. 2026Predict the Retrieval! Test time adaptation for Retrieval Augmented Generationcited by 2
  5. 2026HCAG: Hierarchical Abstraction and Retrieval-Augmented Generation on Theoretical Repositories with LLMscited by 1
  6. 2024An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problemscited by 0
  7. 2025ExpertGenQA: Open-ended QA generation in Specialized Domainscited by 0
  8. 2025XGen-Q: An Explainable Domain-Adaptive LLM Framework with Retrieval-Augmented Generation for Software Securitycited by 0
  9. 2026Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognitioncited by 0
  10. 2026Domain-Adapted Retrieval for In-Context Annotation of Pedagogical Dialogue Actscited by 0
  11. 2026Feedback Adaptation for Retrieval-Augmented Generationcited by 0
  12. 2026BiRD: A Bidirectional Ranking Defense Mechanism for Retrieval Augmented Generationcited by 0
  13. 2026Train, Retrieve, or Both? A Four-Arm Head-to-Head for Correct Statutory Citation on the Ontario Residential Tenancies Actcited by 0
  14. 2026RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generationcited by 0
  15. 2026From Reading Code to Reading Spec: A Verified Layer for LLM-Driven Codebase Maintenancecited by 0
  16. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Domain-Specific Data Generation Framework for RAG Adaptation
  2. 2025Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generation
  3. 2025On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems
  4. 2022Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering
  5. 2025QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance
  6. 2024Retrieval-Augmented Generation for Domain-Specific Question Answering: A Case Study on Pittsburgh and CMU
  7. 2024DuetRAG: Collaborative Retrieval-Augmented Generation
  8. 2024RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation