mlsanityData to 7 Oct 2026

ZeroSearch: Incentivize the Search Capability of LLMs without Searching

arXiv 2505.04588 ↗First posted 7 May 2025cs.CLcited by 203

Not on the map. Why?

Cited by 131 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Effective information searching is essential for enhancing the reasoning and generation capabilities of large language models (LLMs). Recent research has explored using reinforcement learning (RL) to improve LLMs' search capabilities by interacting with live search engines in real-world environments. While these approaches show promising results, they face two major challenges: (1) Uncontrolled Document Quality: The quality of documents returned by search engines is often unpredictable, introducing noise and instability into the training process. (2) Prohibitively High API Costs: RL training requires frequent rollouts, potentially involving hundreds of thousands of search requests, which incur substantial API expenses and severely constrain scalability. To address these challenges, we introduce ZeroSearch, a novel RL framework that incentivizes the capabilities of LLMs to use a real search engine with simulated searches during training. Our approach begins with lightweight supervised fine-tuning to transform the LLM into a retrieval module capable of generating both useful and noisy documents in response to a query. During RL training, we employ a curriculum-based rollout strategy that incrementally degrades the quality of generated documents, progressively eliciting the model's reasoning ability by exposing it to increasingly challenging retrieval scenarios. Extensive experiments demonstrate that ZeroSearch effectively incentivizes the search capabilities of LLMs using a 3B LLM as the retrieval module. Remarkably, a 7B retrieval module achieves comparable performance to the real search engine, while a 14B retrieval module even surpasses it. Furthermore, it generalizes well across both base and instruction-tuned models of various parameter sizes and is compatible with a wide range of RL algorithms.

Key facts. Cited by 203 papers in the map (data through 7 Oct 2026). Builds on 42 of its 49 references that are in the map. First posted on arXiv 7 May 2025.

Builds on

42 of 49 references.

References · 42

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2023A Survey of Large Language Modelscited by 2,537
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  11. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  12. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
30 more
  1. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  2. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  3. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  4. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  5. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  6. 2022Galactica: A Large Language Model for Sciencecited by 736
  7. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  8. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  9. 2023Active Retrieval Augmented Generationcited by 506
  10. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  11. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  12. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
  13. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  14. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  15. 2022Teaching language models to support answers with verified quotescited by 290
  16. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  17. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 268
  18. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  19. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  20. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 142
  21. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
  22. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 89
  23. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  24. 2024Evaluating Mathematical Reasoning Beyond Accuracycited by 69
  25. 2024Enhancing LLM Reasoning with Reward-guided Tree Searchcited by 58
  26. 2024RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinementcited by 38
  27. 2024Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Taskscited by 22
  28. 2024RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generationcited by 20
  29. 2023LPML: LLM-Prompting Markup Language for Mathematical Reasoningcited by 15
  30. 2025AirRAG: Autonomous Strategic Planning and Reasoning Steer Retrieval Augmented Generationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 87; 2026: 116.

203 papers.

Sort:

By area:

  1. 2025Kimi K2: Open Agentic Intelligencecited by 523
  2. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 442
  3. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  4. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  5. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  6. 2025Agentic Reinforced Policy Optimizationcited by 166
  7. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 162
  8. 2025StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimizationcited by 82
  9. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 82
  10. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 75
  11. 2025Tree Search for LLM Agent Reinforcement Learningcited by 73
  12. 2025Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learningcited by 68
38 more
  1. 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Usecited by 67
  2. 2025WideSearch: Benchmarking Agentic Broad Info-Seekingcited by 66
  3. 2025Deep Research: A Survey of Autonomous Research Agentscited by 63
  4. 2025SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesiscited by 55
  5. 2025Visual Agentic Reinforcement Fine-Tuningcited by 53
  6. 2025Towards General Agentic Intelligence via Environment Scalingcited by 51
  7. 2026Hindsight Credit Assignment for Long-Horizon LLM Agentscited by 51
  8. 2025Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learningcited by 50
  9. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 49
  10. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  11. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 40
  12. 2025Agentic Entropy-Balanced Policy Optimizationcited by 39
  13. 2026Dr. Zero: Self-Evolving Search Agents without Training Datacited by 39
  14. 2026Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Taskscited by 37
  15. 2026Dynamic Dual-Granularity Skill Bank for Agentic RLcited by 33
  16. 2026Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interactioncited by 33
  17. 2025Demystifying Reinforcement Learning in Agentic Reasoningcited by 32
  18. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 31
  19. 2025MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool usecited by 30
  20. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  21. 2025MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learningcited by 26
  22. 2025Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Rewardcited by 25
  23. 2025ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learningcited by 24
  24. 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learningcited by 23
  25. 2025SALT: Step-level Advantage Assignment for Long-horizon Agents via Trajectory Graphcited by 22
  26. 2026AgentOCR: Reimagining Agent History via Optical Self-Compressioncited by 22
  27. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
  28. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 21
  29. 2025AI Scientists Fail Without Strong Implementation Capabilitycited by 20
  30. 2025DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seekingcited by 20
  31. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  32. 2026Proximity-Based Multi-Turn Optimization: Practical Credit Assignment for LLM Agent Trainingcited by 18
  33. 2025From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memorycited by 16
  34. 2025RAG-R1: Incentivizing the Search and Reasoning Capabilities of LLMs through Multi-query Parallelismcited by 15
  35. 2025On Group Relative Policy Optimization Collapse in Agent Search: The Lazy Likelihood-Displacementcited by 15
  36. 2026SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphscited by 15
  37. 2025Open Data Synthesis For Deep Researchcited by 14
  38. 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learningcited by 14

50 most cited of 203 All 203 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025ReZero: Enhancing LLM search ability by trying one-more-time
  2. 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents
  3. 2025SEM: Reinforcement Learning for Search-Efficient Large Language Models
  4. 2025Rethinking On-policy Optimization for Query Augmentation
  5. 2025s3: You Don't Need That Much Data to Train a Search Agent via RL
  6. 2025R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning
  7. 2025Enhancing LLMs' Reasoning-Intensive Multimedia Search Capabilities through Fine-Tuning and Reinforcement Learning
  8. 2026OThink-SRR1: Search, Refine and Reasoning with Reinforced Learning for Large Language Models