# Deep Research Bench: Evaluating AI Web Research Agents

arXiv: https://arxiv.org/abs/2506.06287 · HTML: https://mlsanity.com/paper/2506.06287/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 6 May 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 8 of 8)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [Interactive Fiction Games: A Colossal Adventure](https://mlsanity.com/paper/1909.05398/) (2019)
- [AgentSims: An Open-Source Sandbox for Large Language Model Evaluation](https://mlsanity.com/paper/2308.04026/) (2023)
- [Towards a Realistic Long-Term Benchmark for Open-Web Research Agents](https://mlsanity.com/paper/2409.14913/) (2024)

## Built on by (most cited first, 30 of 32)

- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks](https://mlsanity.com/paper/2508.15804/) (2025)
- [DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis](https://mlsanity.com/paper/2508.20033/) (2025)
- [DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports](https://mlsanity.com/paper/2601.08536/) (2026)
- [Characterizing Deep Research: A Benchmark and Formal Definition](https://mlsanity.com/paper/2508.04183/) (2025)
- [Bench to the Future: A Pastcasting Benchmark for Forecasting Agents](https://mlsanity.com/paper/2506.21558/) (2025)
- [Reinforcement Learning Foundations for Deep Research Systems: A Survey](https://mlsanity.com/paper/2509.06733/) (2025)
- [FinResearchBench: A Logic Tree based Agent-as-a-Judge Evaluation Framework for Financial Research Agents](https://mlsanity.com/paper/2507.16248/) (2025)
- [GeoVista: Web-Augmented Agentic Visual Reasoning for Geolocalization](https://mlsanity.com/paper/2511.15705/) (2025)
- [How Far Are We from Genuinely Useful Deep Research Agents?](https://mlsanity.com/paper/2512.01948/) (2025)
- [DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence](https://mlsanity.com/paper/2509.04499/) (2025)
- [Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?](https://mlsanity.com/paper/2605.19196/) (2026)
- [AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery](https://mlsanity.com/paper/2605.23204/) (2026)
- [MLE-Smith: Scaling MLE Tasks with Automated Multi-Agent Pipeline](https://mlsanity.com/paper/2510.07307/) (2025)
- [DataSTORM: Deep Research on Large-Scale Databases using Exploratory Data Analysis and Data Storytelling](https://mlsanity.com/paper/2604.06474/) (2026)
- [Deep Research Comparator: A Platform For Fine-grained Human Annotations of Deep Research Agents](https://mlsanity.com/paper/2507.05495/) (2025)
- [Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts](https://mlsanity.com/paper/2507.19477/) (2025)
- [DR-Arena: an Automated Evaluation Framework for Deep Research Agents](https://mlsanity.com/paper/2601.10504/) (2026)
- [Deep Research, Shallow Evaluation: A Case Study in Meta-Evaluation for Long-Form QA Benchmarks](https://mlsanity.com/paper/2603.06942/) (2026)
- [Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports](https://mlsanity.com/paper/2510.02190/) (2025)
- [Automating Forecasting Question Generation and Resolution for AI Evaluation](https://mlsanity.com/paper/2601.22444/) (2026)
- [R2-Write: Reflection and Revision for Open-Ended Writing with Deep Reasoning](https://mlsanity.com/paper/2604.03004/) (2026)
- [ADRA-Bank: A Modular Benchmark for Academic Deep Research Agents](https://mlsanity.com/paper/2512.00986/) (2025)
- [Dr-CiK: A Benchmark testing Deep Research for Context-Aided Forecasting](https://mlsanity.com/paper/2605.27904/) (2026)
- [WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting](https://mlsanity.com/paper/2607.18084/) (2026)
- [Improving and Evaluating Open Deep Research Agents](https://mlsanity.com/paper/2508.10152/) (2025)
- [Evaluating Strategic Reasoning in Forecasting Agents](https://mlsanity.com/paper/2604.26106/) (2026)
- [SciResearcher: Scaling Deep Research Agents for Frontier Scientific Reasoning](https://mlsanity.com/paper/2605.01489/) (2026)
- [Re2Math: Benchmarking Theorem Retrieval in Research-Level Mathematics](https://mlsanity.com/paper/2605.09012/) (2026)
- [MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents](https://mlsanity.com/paper/2605.30727/) (2026)

## Abstract (arXiv)

Amongst the most common use cases of modern AI is LLM chat with web search enabled. However, no direct evaluations of the quality of web research agents exist that control for the continually-changing web. We introduce Deep Research Bench, consisting of 89 multi-step web research task instances of varying difficulty across 8 diverse task categories, with the answers carefully worked out by skilled humans. We provide a "RetroSearch" environment with a large frozen set of scraped web pages, and demonstrate that offline "RetroSearch" agents perform comparably to "live web" agents, enabling reliable evaluations of models over time. We provide robust agent tooling and scaffolding to benchmark major LLMs as they are released, including "thinking" models like o3 and Gemini 2.5 Pro. We include automated evaluations of the lengthy agent traces to report progress over time in hallucinations, tool use, and forgetting. Finally, we evaluate the major web research products branded as "Deep Research", "Deep Search", "Search", or "Research." Results are available on a public leaderboard at https://drb.futuresearch.ai/.