mlsanityData to 14 Oct 2026

DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning

arXiv 2505.24332 ↗First posted 30 May 2025cs.CLcited by 23

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Information seeking demands iterative evidence gathering and reflective reasoning, yet large language models (LLMs) still struggle with it in open-web question answering. Existing prompting and supervised fine-tuning (SFT) methods remain fixed by prompt rules or training corpora, and are usually benchmarked only on well-structured wiki sources, limiting real-world adaptability. We introduce WebPuzzle, a 24k-sample training and 275-sample test benchmark that evaluates information seeking on the live internet, across both wiki and open-domain queries. Leveraging 7k WebPuzzle instances, we develop DeepDiver, a reinforcement-learning (RL) framework that cultivates Search Intensity Scaling (SIS)-an emergent ability to escalate search frequency and depth instead of settling on overconfident, under-evidenced answers. With SIS, Qwen2.5-7B-Instruct and Pangu-7B-Reasoner attain performance on real-web tasks comparable to the 671B-parameter DeepSeek-R1. We detail DeepDiver's curriculum from cold-start SFT to a well designed RL procedure, and show that its seeking policy generalized from closed-ended queries to open-ended generation such as long-form writing. Our results advance adaptive information seeking in LLMs and provide a rigorous benchmark for future work.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 39 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

28 of 39 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning25 Mar
  2. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning12 Mar
  3. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning7 Mar
  4. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan
  5. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation19 Sep
  6. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions20 Dec

Other references · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2024Qwen2 Technical Reportcited by 2,120
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  10. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  11. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  12. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
10 more
  1. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  2. 2024Measuring short-form factuality in large language modelscited by 373
  3. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
  4. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  5. 2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systemscited by 164
  6. 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
  7. 2024Enhancing LLM Reasoning with Reward-guided Tree Searchcited by 58
  8. 2024Auto-RAG: Autonomous Retrieval-Augmented Generation for Large Language Modelscited by 43
  9. 2024PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Modelscited by 24
  10. 2024PanGu-π Pro:Rethinking Optimization and Architecture for Tiny Language Modelscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 9.

23 papers.

Sort:

By area:

  1. 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 128
  2. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 127
  3. 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 121
  4. 2025AgentFold: Long-Horizon Web Agents with Proactive Context Managementcited by 103
  5. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
  6. 2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learningcited by 61
  7. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  8. 2025Characterizing Deep Research: A Benchmark and Formal Definitioncited by 30
  9. 2026InternAgent-1.5: A Unified Agentic Framework for Long-Horizon Autonomous Scientific Discoverycited by 27
  10. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 22
  11. 2025Open Data Synthesis For Deep Researchcited by 16
  12. 2025Reinforcement Learning Foundations for Deep Research Systems: A Surveycited by 14
11 more
  1. 2025FlowSearch: Advancing deep research with dynamic structured knowledge flowcited by 10
  2. 2025WebSeer: Training Deeper Search Agents through Reinforcement Learning with Self-Reflectioncited by 7
  3. 2025Interact-RAG: Reason and Interact with the Corpus, Beyond Black-Box Retrievalcited by 6
  4. 2026DLLM Agent: See Farther, Run Fastercited by 6
  5. 2025Interaction as Intelligence: Deep Research With Human-AI Partnershipcited by 5
  6. 2026SRR-Judge: Step-Level Rating and Refinement for Enhancing Search-Integrated Reasoning in Search Agentscited by 5
  7. 2026Deep-Reporter: Deep Research for Grounded Multimodal Long-Form Generationcited by 4
  8. 2026Fairness-Aware and Latency-Controllable Scheduling for Chunked-Prefill LLM Servingcited by 2
  9. 2026SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedbackcited by 1
  10. 2026UIS-Digger: Towards Comprehensive Research Agent Systems for Real-world Unindexed Information Seekingcited by 1
  11. 2026DeepStress: Stress-Testing Deep Search Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL
  2. 2025SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesis
  3. 2026OpenSeeker-v2: Pushing the Limits of Search Agents with Informative and High-Difficulty Trajectories
  4. 2026REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents
  5. 2026Search-GRT: Guided Retrieval Training of Search Agents to Optimize for Complex Question Answering
  6. 2025Fathom-DeepResearch: Unlocking Long Horizon Information Retrieval and Synthesis for SLMs
  7. 2026PrimeSeeker: Capability-Oriented Supervision for Deep Search Agents
  8. 2025Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window