# DeepRAG: Thinking to Retrieve Step by Step for Large Language Models

arXiv: https://arxiv.org/abs/2502.01142 · HTML: https://mlsanity.com/paper/2502.01142/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 3 Feb 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)

## Builds on (most cited first, 20 of 20)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://mlsanity.com/paper/2209.10063/) (2022)
- [Multi-Step Reasoning with Large Language Models, a Survey](https://mlsanity.com/paper/2407.11511/) (2024)
- [Question Decomposition Improves the Faithfulness of Model-Generated Reasoning](https://mlsanity.com/paper/2307.11768/) (2023)
- [Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofitting](https://mlsanity.com/paper/2311.13314/) (2023)
- [Understanding Retrieval Augmentation for Long-Form Question Answering](https://mlsanity.com/paper/2310.12150/) (2023)
- [Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluation](https://mlsanity.com/paper/2402.11493/) (2024)
- [Not All Contexts Are Equal: Teaching LLMs Credibility-aware Generation](https://mlsanity.com/paper/2404.06809/) (2024)
- [Divide-or-Conquer? Which Part Should You Distill Your LLM?](https://mlsanity.com/paper/2402.15000/) (2024)
- [Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering](https://mlsanity.com/paper/2411.11504/) (2024)
- [Auto-RT: Automatic Jailbreak Strategy Exploration for Red-Teaming Large Language Models](https://mlsanity.com/paper/2501.01830/) (2025)

## Built on by (most cited first, 30 of 38)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future](https://mlsanity.com/paper/2504.12328/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [Synergizing RAG and Reasoning: A Systematic Review](https://mlsanity.com/paper/2504.15909/) (2025)
- [Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills](https://mlsanity.com/paper/2512.16301/) (2025)
- [R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning](https://mlsanity.com/paper/2505.23794/) (2025)
- [Generative AI Act II: Test Time Scaling Drives Cognition Engineering](https://mlsanity.com/paper/2504.13828/) (2025)
- [Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering](https://mlsanity.com/paper/2601.11255/) (2026)
- [Towards Better Instruction Following Retrieval Models](https://mlsanity.com/paper/2505.21439/) (2025)
- [DeepTrans: Deep Reasoning Translation via Reinforcement Learning](https://mlsanity.com/paper/2504.10187/) (2025)
- [EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes](https://mlsanity.com/paper/2509.00877/) (2025)
- [MIRAGE: Scaling Test-Time Inference with Parallel Graph-Retrieval-Augmented Reasoning Chains](https://mlsanity.com/paper/2508.18260/) (2025)
- [Chain-of-Thought Poisoning Attacks against R1-based Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2505.16367/) (2025)
- [AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning](https://mlsanity.com/paper/2512.16883/) (2025)
- [PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVR](https://mlsanity.com/paper/2601.18207/) (2026)
- [OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answering](https://mlsanity.com/paper/2602.03707/) (2026)
- [CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search](https://mlsanity.com/paper/2604.17555/) (2026)
- [Reinforcement Fine-Tuning for History-Aware Dense Retriever in RAG](https://mlsanity.com/paper/2602.03645/) (2026)
- [DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoning](https://mlsanity.com/paper/2606.07299/) (2026)
- [TrustRAG: An Information Assistant with Retrieval Augmented Generation](https://mlsanity.com/paper/2502.13719/) (2025)
- [Multi-hop Reasoning via Early Knowledge Alignment](https://mlsanity.com/paper/2512.20144/) (2025)
- [Mitigating Hallucination in Financial Retrieval-Augmented Generation via Fine-Grained Knowledge Verification](https://mlsanity.com/paper/2602.05723/) (2026)
- [MiA-Signature: Approximating Global Activation for Long-Context Understanding](https://mlsanity.com/paper/2605.06416/) (2026)
- [AdaptR1: Reinforcement Learning Based Adaptive Interleaved Thinking in Multi-hop Question Answering](https://mlsanity.com/paper/2605.31062/) (2026)
- [Zero-Shot Event Causality Identification via Multi-source Evidence Fuzzy Aggregation with Large Language Models](https://mlsanity.com/paper/2506.05675/) (2025)
- [Improving LLM Reasoning via Dependency-Aware Query Decomposition and Logic-Parallel Content Expansion](https://mlsanity.com/paper/2510.24390/) (2025)
- [PathFinder: MCTS and LLM Feedback-based Path Selection for Multi-Hop Question Answering](https://mlsanity.com/paper/2512.05336/) (2025)
- [TCM-DiffRAG: Personalized Syndrome Differentiation Reasoning Method for Traditional Chinese Medicine based on Knowledge Graph and Chain of Thought](https://mlsanity.com/paper/2602.22828/) (2026)
- [FuzzAgent: Multi-Agent System for Evolutionary Library Fuzzing](https://mlsanity.com/paper/2605.14431/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have shown remarkable reasoning capabilities, while their practical applications are limited by severe factual hallucinations due to limitations in the timeliness, accuracy, and comprehensiveness of their parametric knowledge. Meanwhile, enhancing retrieval-augmented generation (RAG) with reasoning remains challenging due to ineffective task decomposition and redundant retrieval, which can introduce noise and degrade response quality. In this paper, we propose DeepRAG, a framework that models retrieval-augmented reasoning as a Markov Decision Process (MDP), enabling reasonable and adaptive retrieval. By iteratively decomposing queries, DeepRAG dynamically determines whether to retrieve external knowledge or rely on parametric reasoning at each step. Experiments show that DeepRAG improves retrieval efficiency and boosts answer accuracy by 26.4%, demonstrating its effectiveness in enhancing retrieval-augmented reasoning.