# EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge

arXiv: https://arxiv.org/abs/2606.13120 · HTML: https://mlsanity.com/paper/2606.13120/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 14 of 14)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent](https://mlsanity.com/paper/2508.06600/) (2025)
- [WebWalker: Benchmarking LLMs in Web Traversal](https://mlsanity.com/paper/2501.07572/) (2025)
- [WebShaper: Agentically Data Synthesizing via Information-Seeking Formalization](https://mlsanity.com/paper/2507.15061/) (2025)
- [SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models](https://mlsanity.com/paper/2506.01062/) (2025)
- [DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL](https://mlsanity.com/paper/2509.10446/) (2025)
- [DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents](https://mlsanity.com/paper/2601.20975/) (2026)

## Built on by (most cited first, 3 of 3)

- [Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents](https://mlsanity.com/paper/2608.02097/) (2026)
- [AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worlds](https://mlsanity.com/paper/2608.29397/) (2026)
- [Avoiding What Your Teacher Already Knows: Entity Familiarity for Deep-Search Distillation](https://mlsanity.com/paper/2610.16251/) (2026)

## Abstract (arXiv)

Search Agents -- large language models augmented with search tools -- have intensified the need for future-proof evaluation benchmarks. Existing benchmarks such as BrowseComp rely on static knowledge, making them vulnerable to test-set contamination and parametric memorization. Consequently, models can achieve high scores through fact recall rather than genuine retrieval, obscuring true browsing competence via reasoning shortcuts. In this paper, we introduce EvoBrowseComp, an evolving benchmark of 400 English and 400 Chinese contamination-free complex questions synthesized via live-web traversal. To collect these questions, we design a three-agent collaborative framework: (1) a QA synthesis agent that retrieves fresh knowledge from the live web to synthesize QA pairs; (2) an information filtering agent that filters retrieved knowledge in terms of credibility and popularity to block parametric shortcuts; and (3) a high-level guidance agent that formalizes questions into reasoning graphs to reduce logical redundancy and shortcuts in synthesized QA pairs. Because the framework supports fully automated synthesis, EvoBrowseComp can be regularly updated to prevent data contamination and maintain temporal freshness. Extensive experiments confirm its great difficulty, requiring broad horizontal search. It establishes a scalable paradigm for auto-updatable, high-difficulty benchmarking that keeps pace with both evolving world knowledge and advancing agent capabilities.