mlsanityData to 7 Oct 2026

R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning

arXiv 2503.05592 ↗First posted 7 Mar 2025cs.AI cs.CL cs.IRcited by 322

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Existing Large Reasoning Models (LRMs) have shown the potential of reinforcement learning (RL) to enhance the complex reasoning capabilities of Large Language Models~(LLMs). While they achieve remarkable performance on challenging tasks such as mathematics and coding, they often rely on their internal knowledge to solve problems, which can be inadequate for time-sensitive or knowledge-intensive questions, leading to inaccuracies and hallucinations. To address this, we propose R1-Searcher, a novel two-stage outcome-based RL approach designed to enhance the search capabilities of LLMs. This method allows LLMs to autonomously invoke external search systems to access additional knowledge during the reasoning process. Our framework relies exclusively on RL, without requiring process rewards or distillation for a cold start. % effectively generalizing to out-of-domain datasets and supporting both Base and Instruct models. Our experiments demonstrate that our method significantly outperforms previous strong RAG methods, even when compared to the closed-source GPT-4o-mini.

Key facts. Cited by 322 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 34 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

26 of 34 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Chain-of-Retrieval Augmented Generation24 Jan
  2. 2025ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding14 Jan
  3. 2024RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement17 Dec
  4. 2024Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Tasks2 Oct
  5. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions20 Dec

Other references · 21

  1. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  4. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  6. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  7. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  8. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  9. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  10. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  11. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  12. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
9 more
  1. 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185
  2. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  3. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 142
  4. 2025An Empirical Study on Eliciting and Improving R1-like Reasoning Modelscited by 78
  5. 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Modelscited by 69
  6. 2024SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMscited by 67
  7. 2025Atom of Thoughts for Markov LLM Test-Time Scalingcited by 54
  8. 2024MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledgecited by 20
  9. 2024Multi-Reranker: Maximizing performance of retrieval-augmented generation in the FinanceRAG challengecited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 178; 2026: 144.

322 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
  3. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  4. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  5. 2025Memory in the Age of AI Agentscited by 285
  6. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 218
  7. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  8. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  9. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 180
  10. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  11. 2025Agentic Reinforced Policy Optimizationcited by 166
  12. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 152
38 more
  1. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoningcited by 145
  2. 2025Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RLcited by 144
  3. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  4. 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 122
  5. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 119
  6. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
  7. 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 118
  8. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 114
  9. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learningcited by 107
  10. 2025Agent Lightning: Train ANY AI Agents with Reinforcement Learningcited by 94
  11. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
  12. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  13. 2025StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimizationcited by 82
  14. 2025ReasonIR: Training Retrievers for Reasoning Taskscited by 76
  15. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 75
  16. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 72
  17. 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Usecited by 67
  18. 2025WideSearch: Benchmarking Agentic Broad Info-Seekingcited by 66
  19. 2025Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solvingcited by 62
  20. 2025A Survey on (M)LLM-Based GUI Agentscited by 61
  21. 2025SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesiscited by 55
  22. 2025Visual Agentic Reinforcement Fine-Tuningcited by 53
  23. 2025DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RLcited by 53
  24. 2025Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Managementcited by 51
  25. 2025Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learningcited by 50
  26. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 49
  27. 2025SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?cited by 45
  28. 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agentscited by 43
  29. 2025Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agentscited by 43
  30. 2025Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Trainingcited by 42
  31. 2025DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Searchcited by 41
  32. 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agentscited by 39
  33. 2025Scaling Agents via Continual Pre-trainingcited by 39
  34. 2025Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agentscited by 37
  35. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 36
  36. 2025MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agentscited by 35
  37. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 35
  38. 2025Search and Refine During Think: Facilitating Knowledge Refinement for Improved Retrieval-Augmented Reasoningcited by 34

50 most cited of 322 All 322 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
  2. 2025R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning
  3. 2025R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning
  4. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  5. 2025SSRL: Self-Search Reinforcement Learning
  6. 2025Outcome-based Exploration for LLM Reasoning
  7. 2025RAG-R1: Incentivizing the Search and Reasoning Capabilities of LLMs through Multi-query Parallelism
  8. 2025SEM: Reinforcement Learning for Search-Efficient Large Language Models