mlsanityData to 14 Oct 2026

BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents

arXiv 2504.12516 ↗First posted 16 Apr 2025cs.CLcited by 647

Not on the map. Why?

Cited by 431 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present BrowseComp, a simple yet challenging benchmark for measuring the ability for agents to browse the web. BrowseComp comprises 1,266 questions that require persistently navigating the internet in search of hard-to-find, entangled information. Despite the difficulty of the questions, BrowseComp is simple and easy-to-use, as predicted answers are short and easily verifiable against reference answers. BrowseComp for browsing agents can be seen as analogous to how programming competitions are an incomplete but useful benchmark for coding agents. While BrowseComp sidesteps challenges of a true user query distribution, like generating long answers or resolving ambiguity, it measures the important core capability of exercising persistence and creativity in finding information. BrowseComp can be found at https://github.com/openai/simple-evals.

Key facts. Cited by 647 papers in the map (data through 14 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

18 of 28 references.

References · 18

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  8. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  9. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  10. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  11. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  12. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
6 more
  1. 2019ELI5: Long Form Question Answeringcited by 575
  2. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  3. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  4. 2025Humanity's Last Examcited by 491
  5. 2024Measuring short-form factuality in large language modelscited by 373
  6. 2022Teaching language models to support answers with verified quotescited by 292

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 192; 2026: 455.

647 papers.

Sort:

By area:

  1. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  2. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  3. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  4. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  5. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  6. 2025Memory in the Age of AI Agentscited by 303
  7. 2025TTRL: Test-Time Reinforcement Learningcited by 275
  8. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 248
  9. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 229
  10. 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
  11. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 215
  12. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
38 more
  1. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 200
  2. 2026MiMo-V2-Flash Technical Reportcited by 191
  3. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
  4. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  5. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  6. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 149
  7. 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 148
  8. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 145
  9. 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 141
  10. 2025Towards a Science of Scaling Agent Systemscited by 138
  11. 2026Kimi K3: Open Frontier Intelligencecited by 138
  12. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 136
  13. 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 121
  14. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 119
  15. 2025MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Serverscited by 109
  16. 2026EvoSkill: Automated Skill Discovery for Multi-Agent Systemscited by 107
  17. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 104
  18. 2025AgentFold: Long-Horizon Web Agents with Proactive Context Managementcited by 103
  19. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 95
  20. 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 94
  21. 2025WebExplorer: Explore and Evolve for Training Long-Horizon Web Agentscited by 92
  22. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  23. 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Executioncited by 85
  24. 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agentscited by 84
  25. 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 84
  26. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 81
  27. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 80
  28. 2025Tree Search for LLM Agent Reinforcement Learningcited by 79
  29. 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judgecited by 76
  30. 2025SealQA: Raising the Bar for Reasoning in Search-Augmented Language Modelscited by 75
  31. 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learningcited by 72
  32. 2025Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routerscited by 72
  33. 2025WideSearch: Benchmarking Agentic Broad Info-Seekingcited by 69
  34. 2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agentscited by 68
  35. 2025Deep Research: A Survey of Autonomous Research Agentscited by 66
  36. 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Taskscited by 66
  37. 2025Tongyi DeepResearch Technical Reportcited by 66
  38. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62

50 most cited of 647 All 647 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025BEARCUBS: A benchmark for computer-using web agents
  2. 2025Go-Browse: Training Web Agents with Structured Exploration
  3. 2025Improving and Evaluating Open Deep Research Agents
  4. 2024Beyond Browsing: API-Based Web Agents
  5. 2025WebGames: Challenging General-Purpose Web-Browsing AI Agents
  6. 2025REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites
  7. 2025Can Agent Conquer Web? Exploring the Frontiers of ChatGPT Atlas Agent in Web Games
  8. 2026CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception