mlsanityData to 7 Oct 2026

RARE: Retrieval-Augmented Reasoning Modeling

arXiv 2503.23513 ↗First posted 30 Mar 2025cs.CLcited by 11

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Domain-specific intelligence demands specialized knowledge and sophisticated reasoning for problem-solving, posing significant challenges for large language models (LLMs) that struggle with knowledge hallucination and inadequate reasoning capabilities under constrained parameter budgets. Inspired by Bloom's Taxonomy in educational theory, we propose Retrieval-Augmented Reasoning Modeling (RARE), a novel paradigm that decouples knowledge storage from reasoning optimization. RARE externalizes domain knowledge to retrievable sources and internalizes domain-specific reasoning patterns during training. Specifically, by injecting retrieved knowledge into training prompts with masked losses, RARE transforms learning objectives from rote memorization to contextualized reasoning. It enables models to bypass parameter-intensive memorization and prioritize the development of higher-order cognitive processes. Extensive experiments demonstrate that lightweight RARE-trained models (e.g., Llama-3.1-8B) could achieve state-of-the-art performance, surpassing retrieval-augmented GPT-4 and DeepSeek-R1 up to approximately 20% accuracy. RARE establishes a paradigm shift where maintainable external knowledge bases synergize with compact, reasoning-optimized models, collectively driving more scalable domain-specific intelligence.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 59 of its 99 references that are in the map. First posted on arXiv 30 Mar 2025.

Builds on

59 of 99 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning25 Mar
  2. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning12 Mar
  3. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning7 Mar
  4. 2025HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generation18 Feb
  5. 2025Chain-of-Retrieval Augmented Generation24 Jan
  6. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan
  7. 2024RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Models3 Dec
  8. 2024How Much Can RAG Help the Reasoning of LLM?3 Oct

Other references · 51

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  10. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  11. 2023BloombergGPT: A Large Language Model for Financecited by 923
  12. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
38 more
  1. 2023Active Retrieval Augmented Generationcited by 506
  2. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 487
  3. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  4. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 378
  5. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  6. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  7. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  8. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 256
  9. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  10. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 230
  11. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 211
  12. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 209
  13. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 199
  14. 2023Continual Pre-training of Language Modelscited by 194
  15. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  16. 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 168
  17. 2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systemscited by 162
  18. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  19. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  20. 2023S-LoRA: Serving Thousands of Concurrent LoRA Adapterscited by 146
  21. 2023Retrieval meets Long Context Large Language Modelscited by 114
  22. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 113
  23. 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Modelscited by 91
  24. 2024Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Modelscited by 64
  25. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
  26. 2024Enhancing LLM Reasoning with Reward-guided Tree Searchcited by 58
  27. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 57
  28. 2024Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problemscited by 49
  29. 2024Memory3: Language Modeling with Explicit Memorycited by 39
  30. 2025Parametric Retrieval Augmented Generationcited by 38
  31. 2022CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweetscited by 29
  32. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  33. 2023Fin-Fact: A Benchmark Dataset for Multimodal Financial Fact Checking and Explanation Generationcited by 25
  34. 2024Towards Reliable Medical Question Answering: Techniques and Challenges in Mitigating Hallucinations in Language Modelscited by 15
  35. 2025Towards Omni-RAG: Comprehensive Retrieval-Augmented Generation for Large Language Models in Medical Applicationscited by 13
  36. 2025Benchmarking Retrieval-Augmented Generation in Multi-Modal Contextscited by 12
  37. 2024RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generationcited by 5
  38. 2025SSMLoRA: Enhancing Low-Rank Adaptation with State Space Modelcited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 5.

11 papers.

  1. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  2. 2025Search and Refine During Think: Facilitating Knowledge Refinement for Improved Retrieval-Augmented Reasoningcited by 34
  3. 2025Synergizing RAG and Reasoning: A Systematic Reviewcited by 27
  4. 2025Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimizationcited by 5
  5. 2025Reasoning-CV: Fine-tuning Powerful Reasoning LLMs for Knowledge-Assisted Claim Verificationcited by 5
  6. 2026AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generationcited by 5
  7. 2026CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligencecited by 5
  8. 2025HetaRAG: Hybrid Deep Retrieval-Augmented Generation across Heterogeneous Data Storescited by 3
  9. 2026GroupRAG: Cognitively Inspired Group-Aware Retrieval and Reasoning via Knowledge-Driven Problem Structuringcited by 0
  10. 2026AGE: Adaptive-masking for Graph Embedding in Graph Retrieval-Augmented Generationcited by 0
  11. 2026ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scalingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When to Retrieve During Reasoning: Adaptive Retrieval for Large Reasoning Models
  2. 2025UR2: Unify RAG and Reasoning through Reinforcement Learning
  3. 2026Adaptive Retrieval helps Reasoning in LLMs -- but mostly if it's not used
  4. 2025Big Reasoning with Small Models: Instruction Retrieval at Inference Time
  5. 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks
  6. 2025RaDeR: Reasoning-aware Dense Retrieval Models
  7. 2025Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning
  8. 2025Improving Context Fidelity via Native Retrieval-Augmented Reasoning