mlsanityData to 14 Oct 2026

Self-Routing RAG: Binding Selective Retrieval with Knowledge Verbalization

arXiv 2504.01018 ↗First posted 1 Apr 2025cs.CLcited by 9

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Selective retrieval aims to make retrieval-augmented generation (RAG) more efficient and reliable by skipping retrieval when an LLM's parametric knowledge suffices. Despite promising results, existing methods are constrained by a binary design choice: either retrieve from a single external source or skip retrieval and let the LLM directly produce the final answer. We argue that this fallback underestimates the model's knowledge and obscures the more general multi-source decision problem that arises in practical systems. We propose Self-Routing RAG (SR-RAG), which casts selective retrieval as knowledge source selection and treats the LLM itself as a first-class knowledge source. SR-RAG learns to select an appropriate knowledge source, optionally verbalize parametric knowledge, and answer using the selected source, all within a single left-to-right generation pass. SR-RAG further augments source selection by combining LLM-based uncertainty with a flexible external policy datastore to improve decision calibration. Across four benchmarks and three 7B-class LLMs, SR-RAG outperforms a strong selective retrieval baseline by 8.5%/2.1%/4.7% while performing 26%/40%/21% fewer retrievals, and it achieves favorable accuracy-latency trade-offs without dataset-specific threshold tuning.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 45 of its 52 references that are in the map. First posted on arXiv 1 Apr 2025.

Builds on

45 of 52 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home22 Jan
  2. 2024SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation27 Jun
  3. 2024Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation19 Jun
  4. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity21 Mar
  5. 2024RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedback11 Mar
  6. 2023Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models23 Oct
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 38

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  10. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  11. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  12. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
26 more
  1. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  2. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  4. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  5. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  6. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  7. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  8. 2023Active Retrieval Augmented Generationcited by 516
  9. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 346
  10. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
  11. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  12. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 234
  13. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 214
  14. 2024Corrective Retrieval Augmented Generationcited by 197
  15. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  16. 2024Distilling System 2 into System 1cited by 159
  17. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 143
  18. 2021Efficient Nearest Neighbor Language Modelscited by 93
  19. 2021Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challengecited by 80
  20. 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Modelscited by 69
  21. 2024DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Modelscited by 63
  22. 2024Repoformer: Selective Retrieval for Repository-Level Code Completioncited by 55
  23. 2024Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimensioncited by 45
  24. 2024PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makerscited by 42
  25. 2023Interpretable Unified Language Checkingcited by 26
  26. 2024Dynamic Strategy Planning for Efficient Question Answering with Large Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 7.

9 papers.

  1. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 31
  2. 2026RAGRouter-Bench: A Dataset and Benchmark for Adaptive RAG Routingcited by 6
  3. 2025RAG in the Wild: On the (In)effectiveness of LLMs with Mixture-of-Knowledge Retrieval Augmentationcited by 3
  4. 2026User as Engram: Internalizing Per-User Memory as Local Parametric Editscited by 3
  5. 2026The Anatomy of Uncertainty in LLMscited by 2
  6. 2026Modality Relevance is not Modality Utility: Post-hoc Selective Modality Escalation for Cost-Aware Multimodal RAGcited by 1
  7. 2026Dual-Layer Agentic Memory with Fast Write Routing and Slow Consolidationcited by 1
  8. 2026Did You Check the Right Pocket? Cost-Sensitive Store Routing for Memory-Augmented Agentscited by 0
  9. 2026KACE: Knowledge-Adaptive Context Engineering for Mathematical Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Optimizing Knowledge Integration in Retrieval-Augmented Generation with Self-Selection
  2. 2024Auto-RAG: Autonomous Retrieval-Augmented Generation for Large Language Models
  3. 2025SKILL-RAG: Self-Knowledge Induced Learning and Filtering for Retrieval-Augmented Generation
  4. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  5. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  6. 2024PrefRAG: Preference-Driven Multi-Source Retrieval Augmented Generation
  7. 2026Self-Correcting RAG: Enhancing Faithfulness via MMKP Context Selection and NLI-Guided MCTS
  8. 2025Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing