LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know?
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Are LLM-based search agents genuinely searching, or using the web to verify what they already know? We study this question on BrowseComp with three diagnostics. Our analysis reveals Intrinsic Knowledge Dependence (IKD): even with tool access, agents often rely on intrinsic knowledge -- information encoded in the model before retrieval -- rather than on external evidence. Agents answer up to 44.5% of BrowseComp questions without tools, generate more than half of their search queries from internally produced hypotheses rather than retrieved leads, and perform worse than closed-book baselines when answer-supporting evidence is removed. These results suggest that static search benchmarks can reward memory-backed verification rather than evidence-driven discovery, conflating what agents already know with what they can find. We then introduce LiveBrowseComp, a deep-search benchmark designed to evaluate agents beyond intrinsic coverage. It contains 335 human-authored questions whose answers depend on facts published within the 90 days preceding benchmark construction, drawn from six updated sources and filtered to exclude globally salient events. On LiveBrowseComp, all evaluated agents fall below 2% closed-book accuracy, search-augmented scores drop by 25-40 points relative to BrowseComp, and prior model rankings no longer reliably predict performance. LiveBrowseComp is available at https://huggingface.co/datasets/Forival/LiveBrowseComp.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 46 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
30 of 46 references.
Closest ancestors · 8
Same direction, newest first
- 2026REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents15 Feb
- 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents28 Jan
- 2025WideSearch: Benchmarking Agentic Broad Info-Seeking11 Aug
- 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent8 Aug
- 2025An Illusion of Progress? Assessing the Current State of Web Agents2 Apr
- 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul
- 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
Other references · 22
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Qwen3 Technical Reportcited by 8,195
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
- 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
10 more
- 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
- 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
- 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
- 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
- 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 139
- 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 136
- 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluationcited by 73
- 2024Are Your LLMs Capable of Stable Reasoning?cited by 65
- 2023Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Modelscited by 46
- 2024Context versus Prior Knowledge in Language Modelscited by 25
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026EigentSearch-Q+: Enhancing Deep Research Agents with Structured Reasoning Tools
- 2026Projecting BrowseComp-Plus onto ClimbMix: Toward More Realistic Corpora for Agentic Search
- 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents
- 2026Question's Gambit: The First Move Matters in Agentic Deep Search
- 2026BrowseComp-V3: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents
- 2025Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agents
- 2026Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wiki
- 2026EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge