DRBench: A Realistic Benchmark for Enterprise Deep Research
Not on the map. Why?
Abstract (arXiv)
We introduce DRBench, a benchmark for evaluating AI agents on complex, open-ended deep research tasks in enterprise settings. Unlike prior benchmarks that focus on simple questions or web-only queries, DRBench evaluates agents on multi-step queries (for example, "What changes should we make to our product roadmap to ensure compliance with this standard?") that require identifying supporting facts from both the public web and private company knowledge base. Each task is grounded in realistic user personas and enterprise context, spanning a heterogeneous search space that includes productivity software, cloud file systems, emails, chat conversations, and the open web. Tasks are generated through a carefully designed synthesis pipeline with human-in-the-loop verification, and agents are evaluated on their ability to recall relevant insights, maintain factual accuracy, and produce coherent, well-structured reports. We release 100 deep research tasks across 10 domains, such as Sales, Cybersecurity, and Compliance. We demonstrate the effectiveness of DRBench by evaluating diverse DR agents across open- and closed-source models (such as GPT, Llama, and Qwen) and DR strategies, highlighting their strengths, weaknesses, and the critical path for advancing enterprise deep research. Code and data are available at https://github.com/ServiceNow/drbench.
Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Sep 2025.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2025: 2; 2026: 17.
19 papers.
Sort:
By area:
- 2025Enterprise Deep Research: Steerable Multi-Agent Deep Research for Enterprise Analyticscited by 20
- 2026Calibrate-Then-Act: Cost-Aware Exploration in LLM Agentscited by 16
- 2025How Far Are We from Genuinely Useful Deep Research Agents?cited by 11
- 2026DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivitycited by 11
- 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discoverycited by 9
- 2026Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectoriescited by 6
- 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectorycited by 5
- 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcomecited by 5
- 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learningcited by 1
- 2026NGDBench: Towards Neural Graph Data Managementcited by 0
- 2026MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agentscited by 0
- 2026Multi-Turn Evaluation of Deep Research Agents Under Process-Level Feedbackcited by 0
7 more
- 2026DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Taskscited by 0
- 2026MCompassRAG: Topic Metadata as a Semantic Compass for Paragraph-Level Retrievalcited by 0
- 2026Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generationcited by 0
- 2026DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agentscited by 0
- 2026WinSyn: An Automated Pipeline for Realistic Enterprise Question-Answering Evaluationcited by 0
- 2026"Nothing to See Here'': Unintended Disclosure through Revision Traces of LLM Deliverablescited by 0
- 2026DeepRewind: Predicting and Repairing Premature Commitments in Deep Research Agentscited by 0
- 2026DeepRewind: Predicting and Repairing Premature Commitments in Deep Research Agents28 Sep 2026
- 2026"Nothing to See Here'': Unintended Disclosure through Revision Traces of LLM Deliverables28 Sep 2026
- 2026WinSyn: An Automated Pipeline for Realistic Enterprise Question-Answering Evaluation10 Sep 2026
- 2026DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agents4 Sep 2026
- 2026Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generation1 Jul 2026
- 2026MCompassRAG: Topic Metadata as a Semantic Compass for Paragraph-Level Retrieval16 Jun 2026
- 2026DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks11 Jun 2026
- 2026Multi-Turn Evaluation of Deep Research Agents Under Process-Level Feedback8 Jun 2026
- 2026Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories1 Jun 2026
- 2026MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents29 May 2026
- 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery22 May 2026
- 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome30 Mar 2026
7 more
- 2026NGDBench: Towards Neural Graph Data Management27 Feb 2026
- 2026Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents18 Feb 2026
- 2026DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivity12 Feb 2026
- 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learning3 Feb 2026
- 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory30 Jan 2026
- 2025How Far Are We from Genuinely Useful Deep Research Agents?1 Dec 2025
- 2025Enterprise Deep Research: Steerable Multi-Agent Deep Research for Enterprise Analytics20 Oct 2025
- 2026Calibrate-Then-Act: Cost-Aware Exploration in LLM Agentscited by 16
- 2025How Far Are We from Genuinely Useful Deep Research Agents?cited by 11
- 2026Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectoriescited by 6
- 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectorycited by 5
- 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcomecited by 5
- 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learningcited by 1
- 2026MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agentscited by 0
- 2026Multi-Turn Evaluation of Deep Research Agents Under Process-Level Feedbackcited by 0
- 2026DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Taskscited by 0
- 2026DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agentscited by 0
- 2026DeepRewind: Predicting and Repairing Premature Commitments in Deep Research Agentscited by 0
- 2026DeepRewind: Predicting and Repairing Premature Commitments in Deep Research Agents28 Sep 2026
- 2026DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agents4 Sep 2026
- 2026DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks11 Jun 2026
- 2026Multi-Turn Evaluation of Deep Research Agents Under Process-Level Feedback8 Jun 2026
- 2026Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories1 Jun 2026
- 2026MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents29 May 2026
- 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome30 Mar 2026
- 2026Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents18 Feb 2026
- 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learning3 Feb 2026
- 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory30 Jan 2026
- 2025How Far Are We from Genuinely Useful Deep Research Agents?1 Dec 2025
- 2025Enterprise Deep Research: Steerable Multi-Agent Deep Research for Enterprise Analyticscited by 20
- 2026DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivitycited by 11
- 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discoverycited by 9
- 2026NGDBench: Towards Neural Graph Data Managementcited by 0
- 2026"Nothing to See Here'': Unintended Disclosure through Revision Traces of LLM Deliverablescited by 0
- 2026"Nothing to See Here'': Unintended Disclosure through Revision Traces of LLM Deliverables28 Sep 2026
- 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery22 May 2026
- 2026NGDBench: Towards Neural Graph Data Management27 Feb 2026
- 2026DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivity12 Feb 2026
- 2025Enterprise Deep Research: Steerable Multi-Agent Deep Research for Enterprise Analytics20 Oct 2025