How Far Are We from Genuinely Useful Deep Research Agents?
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Deep Research Agents (DRAs) aim to automatically produce analyst-level reports through iterative information retrieval and synthesis. However, most existing DRAs were validated on question-answering benchmarks, while research on generating comprehensive reports remains overlooked. Worse, current benchmarks for report synthesis suffer from task complexity and subjective metrics -- this fails to reflect user demands and limits the practical utility of generated reports. To address these gaps, we present Fine-grained DEepResearch bench (FINDER), an enhanced benchmark consisting of 100 human-curated research tasks with 419 structured checklist items that standardize report structure, analytical depth, and factual grounding. Based on approximately 1,000 reports produced by mainstream DRAs, we further propose Deep rEsearch Failure Taxonomy (DEFT), the first failure taxonomy for deep research agents. DEFT contains 14 fine-grained failure modes across reasoning, retrieval, and generation, and is built upon grounded theory with human-LLM co-annotating and inter-annotator reliability validation. Our experimental findings reveal that current DRAs struggle not with task comprehension but with evidence integration, verification, and reasoning-resilient planning.
Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 55 references that are in the map. First posted on arXiv 1 Dec 2025.
Builds on
29 of 55 references.
Closest ancestors · 8
Same direction, newest first
- 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling14 Nov
- 2025Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Execution29 Sep
- 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent8 Aug
- 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge26 Jun
- 2025Deep Research Agents: A Systematic Examination And Roadmap22 Jun
- 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents13 Jun
- 2025DeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research25 May
- 2025Deep Research Bench: Evaluating AI Web Research Agents6 May
Other references · 21
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
- 2025Humanity's Last Examcited by 467
- 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
- 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 171
- 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automationcited by 129
- 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 120
- 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 105
- 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 90
- 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 75
9 more
- 2025Scaling Test-time Compute for LLM Agentscited by 59
- 2025A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applicationscited by 48
- 2025Efficient Agents: Building Effective Agents While Reducing Costcited by 32
- 2025DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesiscited by 31
- 2025Characterizing Deep Research: A Benchmark and Formal Definitioncited by 29
- 2025DRBench: A Realistic Benchmark for Enterprise Deep Researchcited by 19
- 2021Socio-Technical Grounded Theory for Software Engineeringcited by 7
- 2025QSAF: A Novel Mitigation Framework for Cognitive Degradation in Agentic AIcited by 6
- 2025A2FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoningcited by 6
Built on by
Citing papers per year: 2026: 11.
11 papers.
- 2026Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verificationcited by 19
- 2026Deep Research, Shallow Evaluation: A Case Study in Meta-Evaluation for Long-Form QA Benchmarkscited by 4
- 2026Can Deep Research Agents Retrieve and Organize? Evaluating the Synthesis Gap with Expert Taxonomiescited by 3
- 2026Mind2Report: Expert-Level Commercial Report Synthesis via Cognitive Deep Research Agentcited by 2
- 2026ScholarGym: Benchmarking Large Language Model Capabilities in the Information-Gathering Stage of Deep Researchcited by 1
- 2026LiveNewsBench: Evaluating LLM Web Search Capabilities with Freshly Curated Newscited by 1
- 2026Mind DeepResearch Technical Reportcited by 0
- 2026From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Taskscited by 0
- 2026On-Device Deep Research at 4B: Exposure Bounds Faithfulness, Retrieval Bounds Coveragecited by 0
- 2026CIVI: A Framework for Diagnosing Search Agent Failures in Civic Informationcited by 0
- 2026YouRA: A Persistent-State Architecture for Evidence-Traceable Autonomous Research Agentscited by 0
- 2026YouRA: A Persistent-State Architecture for Evidence-Traceable Autonomous Research Agents1 Oct 2026
- 2026CIVI: A Framework for Diagnosing Search Agent Failures in Civic Information8 Sep 2026
- 2026On-Device Deep Research at 4B: Exposure Bounds Faithfulness, Retrieval Bounds Coverage14 Jul 2026
- 2026From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks30 Apr 2026
- 2026Mind DeepResearch Technical Report16 Apr 2026
- 2026Deep Research, Shallow Evaluation: A Case Study in Meta-Evaluation for Long-Form QA Benchmarks6 Mar 2026
- 2026LiveNewsBench: Evaluating LLM Web Search Capabilities with Freshly Curated News14 Feb 2026
- 2026ScholarGym: Benchmarking Large Language Model Capabilities in the Information-Gathering Stage of Deep Research29 Jan 2026
- 2026Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification22 Jan 2026
- 2026Can Deep Research Agents Retrieve and Organize? Evaluating the Synthesis Gap with Expert Taxonomies18 Jan 2026
- 2026Mind2Report: Expert-Level Commercial Report Synthesis via Cognitive Deep Research Agent8 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks
- 2026DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports
- 2025Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports
- 2026Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision
- 2025Deep Research: A Survey of Autonomous Research Agents
- 2025Understanding DeepResearch via Reports
- 2026From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents
- 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents