mlsanityData to 7 Oct 2026

How Far Are We from Genuinely Useful Deep Research Agents?

arXiv 2512.01948 ↗First posted 1 Dec 2025cs.CLcited by 11

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Deep Research Agents (DRAs) aim to automatically produce analyst-level reports through iterative information retrieval and synthesis. However, most existing DRAs were validated on question-answering benchmarks, while research on generating comprehensive reports remains overlooked. Worse, current benchmarks for report synthesis suffer from task complexity and subjective metrics -- this fails to reflect user demands and limits the practical utility of generated reports. To address these gaps, we present Fine-grained DEepResearch bench (FINDER), an enhanced benchmark consisting of 100 human-curated research tasks with 419 structured checklist items that standardize report structure, analytical depth, and factual grounding. Based on approximately 1,000 reports produced by mainstream DRAs, we further propose Deep rEsearch Failure Taxonomy (DEFT), the first failure taxonomy for deep research agents. DEFT contains 14 fine-grained failure modes across reasoning, retrieval, and generation, and is built upon grounded theory with human-LLM co-annotating and inter-annotator reliability validation. Our experimental findings reveal that current DRAs struggle not with task comprehension but with evidence integration, verification, and reasoning-resilient planning.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 55 references that are in the map. First posted on arXiv 1 Dec 2025.

Builds on

29 of 55 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling14 Nov
  2. 2025Flash-Searcher: Fast and Effective Web Agents via DAG-Based Parallel Execution29 Sep
  3. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent8 Aug
  4. 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge26 Jun
  5. 2025Deep Research Agents: A Systematic Examination And Roadmap22 Jun
  6. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents13 Jun
  7. 2025DeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research25 May
  8. 2025Deep Research Bench: Evaluating AI Web Research Agents6 May

Other references · 21

  1. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  3. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  4. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  5. 2025Humanity's Last Examcited by 467
  6. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  7. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 171
  8. 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automationcited by 129
  9. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 120
  10. 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 105
  11. 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 90
  12. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 75
9 more
  1. 2025Scaling Test-time Compute for LLM Agentscited by 59
  2. 2025A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applicationscited by 48
  3. 2025Efficient Agents: Building Effective Agents While Reducing Costcited by 32
  4. 2025DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesiscited by 31
  5. 2025Characterizing Deep Research: A Benchmark and Formal Definitioncited by 29
  6. 2025DRBench: A Realistic Benchmark for Enterprise Deep Researchcited by 19
  7. 2021Socio-Technical Grounded Theory for Software Engineeringcited by 7
  8. 2025QSAF: A Novel Mitigation Framework for Cognitive Degradation in Agentic AIcited by 6
  9. 2025A2FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 11.

11 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks
  2. 2026DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports
  3. 2025Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports
  4. 2026Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision
  5. 2025Deep Research: A Survey of Autonomous Research Agents
  6. 2025Understanding DeepResearch via Reports
  7. 2026From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents
  8. 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents