Data to 14 Oct 2026

WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents

arXiv 2509.06501 ↗First posted 8 Sep 2025cs.CLcited by 92

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

The paradigm of Large Language Models (LLMs) has increasingly shifted toward agentic applications, where web browsing capabilities are fundamental for retrieving information from diverse online sources. However, existing open-source web agents either demonstrate limited information-seeking abilities on complex tasks or lack transparent implementations. In this work, we identify that the key challenge lies in the scarcity of challenging data for information seeking. To address this limitation, we introduce WebExplorer: a systematic data generation approach using model-based exploration and iterative, long-to-short query evolution. This method creates challenging query-answer pairs that require multi-step reasoning and complex web navigation. By leveraging our curated high-quality dataset, we successfully develop advanced web agent WebExplorer-8B through supervised fine-tuning followed by reinforcement learning. Our model supports 128K context length and up to 100 tool calling turns, enabling long-horizon problem solving. Across diverse information-seeking benchmarks, WebExplorer-8B achieves the state-of-the-art performance at its scale. Notably, as an 8B-sized model, WebExplorer-8B is able to effectively search over an average of 16 turns after RL training, achieving higher accuracy than WebSailor-72B on BrowseComp-en/zh and attaining the best performance among models up to 100B parameters on WebWalkerQA and FRAMES. Beyond these information-seeking tasks, our model also achieves strong generalization on the HLE benchmark even though it is only trained on knowledge-intensive QA data. These results highlight our approach as a practical path toward long-horizon web agents.

Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 32 references that are in the map. First posted on arXiv 8 Sep 2025.

Builds on

22 of 32 references.

Closest ancestors · 5

Same direction, newest first

Other references · 17

2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
2025Qwen3 Technical Reportcited by 8,820
2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
2023GAIA: a benchmark for General AI Assistantscited by 1,128
2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
2025Humanity's Last Examcited by 491
2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
2025WebWalker: Benchmarking LLMs in Web Traversalcited by 175

5 more

2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 162
2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 121
2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 80
2023Calibrating LLM-Based Evaluatorcited by 53
2025SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyondcited by 50

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 67.

92 papers.

Sort:

By area:

2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 141
2025AgentFold: Long-Horizon Web Agents with Proactive Context Managementcited by 103
2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 81
2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agentscited by 68
2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learningcited by 61
2026The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligencecited by 61
2025DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RLcited by 55
2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
2025Budget-Aware Tool Use Enables Effective Agent Scalingcited by 44
2025Scaling Agents via Continual Pre-trainingcited by 40
2026OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesiscited by 37

38 more

2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 22
2026Terminal-World: Scaling Terminal-Agent Environments via Agent Skillscited by 20
2025Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Windowcited by 19
2025WebLeaper: Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seekingcited by 18
2026Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generationcited by 18
2025BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actionscited by 17
2026Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Designcited by 16
2025ParallelMuse: Agentic Parallel Thinking for Deep Information Seekingcited by 14
2026Chaining the Evidence: Robust Reinforcement Learning for Deep Search Agents with Citation-Aware Rubric Rewardscited by 14
2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Usecited by 13
2025Pushing Test-Time Scaling Limits of Deep Search with Asymmetric Verificationcited by 12
2025Lost in the Maze: Overcoming Context Limitations in Long-Horizon Agentic Searchcited by 12
2025InteractComp: Evaluating Search Agents With Ambiguous Queriescited by 12
2026OffSeeker: Online Reinforcement Learning Is Not All You Need for Deep Research Agentscited by 12
2026RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewardscited by 12
2026Self-Compacting Language Model Agentscited by 11
2025Which Heads Matter for Reasoning? RL-Guided KV Cache Compressioncited by 10
2026LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agentcited by 10
2025PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoningcited by 9
2026AgentSwing: Adaptive Parallel Context Management Routing for Long-Horizon Web Agentscited by 9
2026WebAnchor: Anchoring Agent Planning to Stabilize Long-Horizon Web Reasoningcited by 8
2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
2025Reinforcement Learning for Tool-Integrated Interleaved Thinking towards Cross-Domain Generalizationcited by 7
2026Revisiting Text Ranking in Deep Researchcited by 7
2026SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Taskscited by 7
2025FoldAct: Efficient and Stable Context Folding for Long-Horizon Search Agentscited by 6
2025Nested Browser-Use Learning for Agentic Information Seekingcited by 6
2026ColorBrowserAgent: Complex Long-Horizon Browser Agent with Adaptive Knowledge Evolutioncited by 6
2026Argus: Evidence Assembly for Scalable Deep Research Agentscited by 6
2026When Is Enough Not Enough? Illusory Completion in Search Agentscited by 5
2026SRR-Judge: Step-Level Rating and Refinement for Enhancing Search-Integrated Reasoning in Search Agentscited by 5
2026Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalizationcited by 5
2026DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Datacited by 5
2026S1-DeepResearch: Beyond Search, Toward Real-World Long-Horizon Research Agentscited by 5
2026STAMP: Provenance-Guided Credit Assignment for Deep Search Agentscited by 5
2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
2025Synthesizing Agentic Data for Web Agents with Progressive Difficulty Enhancement Mechanismscited by 4

50 most cited of 92 All 92 (JSON)

Similar, not cited

Close in wording, no citation either way