mlsanityData to 7 Oct 2026

EvolveSearch: An Iterative Self-Evolving Search Agent

arXiv 2505.22501 ↗First posted 28 May 2025cs.CLcited by 42

On the map: LLM agents and tool use → Web, search, and research agents

also in 1 more
Abstract (arXiv)

The rapid advancement of large language models (LLMs) has transformed the landscape of agentic information seeking capabilities through the integration of tools such as search engines and web browsers. However, current mainstream approaches for enabling LLM web search proficiency face significant challenges: supervised fine-tuning struggles with data production in open-search domains, while RL converges quickly, limiting their data utilization efficiency. To address these issues, we propose EvolveSearch, a novel iterative self-evolution framework that combines SFT and RL to enhance agentic web search capabilities without any external human-annotated reasoning data. Extensive experiments on seven multi-hop question-answering (MHQA) benchmarks demonstrate that EvolveSearch consistently improves performance across iterations, ultimately achieving an average improvement of 4.7% over the current state-of-the-art across seven benchmarks, opening the door to self-evolution agentic capabilities in open web search domains.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 20 of its 35 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

20 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan

Other references · 19

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  5. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  7. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  8. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  9. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  10. 2022Large Language Models Can Self-Improvecited by 656
  11. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  12. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 403
7 more
  1. 2023Fine-tuning Language Models for Factualitycited by 219
  2. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  3. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 171
  4. 2019Revisiting Self-Training for Neural Sequence Generationcited by 154
  5. 2023Agents: An Open-source Framework for Autonomous Language Agentscited by 116
  6. 2023Post Hoc Explanations of Language Models Can Improve Language Modelscited by 50
  7. 2024Plausible Extractive Rationalization through Semi-Supervised Entailment Signalcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 22; 2026: 20.

42 papers.

Sort:

By area:

  1. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 218
  2. 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 118
  3. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 75
  4. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 68
  5. 2025Deep Research: A Survey of Autonomous Research Agentscited by 63
  6. 2025Scaling Agents via Continual Pre-trainingcited by 39
  7. 2025Agentic Entropy-Balanced Policy Optimizationcited by 39
  8. 2026Dr. Zero: Self-Evolving Search Agents without Training Datacited by 39
  9. 2025Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Executioncited by 27
  10. 2026ExpSeek: Self-Triggered Experience Seeking for Web Agentscited by 23
  11. 2025Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Windowcited by 19
  12. 2025BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actionscited by 17
30 more
  1. 2026Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gaincited by 16
  2. 2025ParallelMuse: Agentic Parallel Thinking for Deep Information Seekingcited by 14
  3. 2025Reinforcement Learning Foundations for Deep Research Systems: A Surveycited by 13
  4. 2025Alita-G: Self-Evolving Generative Agent for Agent Generationcited by 10
  5. 2025Repurposing Synthetic Data for Fine-grained Search Agent Supervisioncited by 10
  6. 2026SkillMaster: Toward Autonomous Skill Mastery in LLM Agentscited by 8
  7. 2025LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Servicescited by 7
  8. 2025Erase to Improve: Erasable Reinforcement Learning for Search-Augmented LLMscited by 6
  9. 2025A2FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoningcited by 6
  10. 2025WebSeer: Training Deeper Search Agents through Reinforcement Learning with Self-Reflectioncited by 6
  11. 2025DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shapingcited by 5
  12. 2025Deep Literature Survey Automation with an Iterative Workflowcited by 5
  13. 2026DLLM Agent: See Farther, Run Fastercited by 5
  14. 2026AutoAgent: Evolving Cognition and Elastic Memory Orchestration for Adaptive Agentscited by 5
  15. 2026Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalizationcited by 4
  16. 2025CubeBench: Diagnosing Interactive, Long-Horizon Spatial Reasoning Under Partial Observationscited by 3
  17. 2026GeoBrowse: A Geolocation Benchmark for Agentic Tool Use with Expert-Annotated Reasoning Tracescited by 3
  18. 2025Rethinking the Design of Reinforcement Learning-Based Deep Research Agentscited by 2
  19. 2026Self-evolving Embodied AIcited by 2
  20. 2026Knowledge-Graph Paths as Intermediate Supervision for Self-Evolving Search Agentscited by 2
  21. 2026Self-Evolving Deep Research via Joint Generation and Evaluationcited by 2
  22. 2026ScholarGym: Benchmarking Large Language Model Capabilities in the Information-Gathering Stage of Deep Researchcited by 1
  23. 2026Can LLMs Time Travel? Enhancing Temporal Consistency in Legal Agentic Search through Reinforcement Learningcited by 1
  24. 2026CAFE: Self-Improving Search Agents Need Co-Evolving Feedbackcited by 1
  25. 2026Beyond Relevance: Utility-Centric Retrieval in the LLM Eracited by 0
  26. 2026An Interactive Paradigm for Deep Researchcited by 0
  27. 2026When Rules Learn: A Self-Evolving Agent for Legal Case Retrievalcited by 0
  28. 2026STEC: Evidence Compression for Deep Search in Open-domain Multi-Hop QAcited by 0
  29. 2026Dr. Free: You Don't Need Difficulty Rewards for Self-Evolving Search Agentscited by 0
  30. 2026False Frontiers: Diagnosing and Mitigating Co-Cheating in Self-Evolving Search Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers
  2. 2025Supervising the search process produces reliable and generalizable information-seeking agents
  3. 2026DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment
  4. 2025RE-Searcher: Robust Agentic Search with Goal-oriented Planning and Self-reflection
  5. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
  6. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents
  7. 2025RAVine: Reality-Aligned Evaluation for Agentic Search
  8. 2025InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented Generation