mlsanityData to 14 Oct 2026

Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision

arXiv 2601.13217 ↗First posted 19 Jan 2026cs.CL cs.AIcited by 4

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Existing benchmarks for Deep Research Agents (DRAs) treat report generation as a single-shot writing task, which fundamentally diverges from how human researchers iteratively draft and revise reports via self-reflection or peer feedback. Whether DRAs can reliably revise reports with user feedback remains unexplored. We introduce Mr Dre, an evaluation suite that establishes multi-turn report revision as a new evaluation axis for DRAs. Mr Dre consists of (1) a unified long-form report evaluation protocol spanning comprehensiveness, factuality, and presentation, and (2) a human-verified feedback simulation pipeline for multi-turn revision. Our analysis of five diverse DRAs reveals a critical limitation: while agents can address most user feedback, they also regress on 16-27% of previously covered content and citation quality. Over multiple revision turns, even the best-performing agents leave significant headroom, as they continue to disrupt content outside the feedback's scope and fail to preserve earlier edits. We further show that these issues are not easily resolvable through inference-time fixes such as prompt engineering and a dedicated sub-agent for report revision.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 58 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

32 of 58 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents10 Nov
  2. 2025LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild16 Oct
  3. 2025ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks14 Aug
  4. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent8 Aug
  5. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents13 Jun

Other references · 27

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2025s1: Simple test-time scalingcited by 1,341
  6. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  7. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  10. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  11. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  12. 2019ELI5: Long Form Question Answeringcited by 575
15 more
  1. 2023Active Retrieval Augmented Generationcited by 516
  2. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
  3. 2023Evaluating Verifiability in Generative Search Enginescited by 303
  4. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 148
  5. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 117
  6. 2024CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklistscited by 94
  7. 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 87
  8. 2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agentscited by 68
  9. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 45
  10. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answeringcited by 43
  11. 2025RocketEval: Efficient Automated LLM Evaluation via Grading Checklistcited by 39
  12. 2025ResearcherBench: Evaluating Deep AI Research Systems on the Frontiers of Scientific Inquirycited by 32
  13. 2024Learning to Refine with Fine-Grained Natural Language Feedbackcited by 27
  14. 2025ExpertLongBench: Benchmarking Language Models on Expert-Level Long-Form Generation Tasks with Structured Checklistscited by 24
  15. 2023MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025How Far Are We from Genuinely Useful Deep Research Agents?
  2. 2026DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing
  3. 2026DR3-Eval: Towards Realistic and Reproducible Deep Research Evaluation
  4. 2026AgentCPM-Report: Interleaving Drafting and Deepening for Open-Ended Deep Research
  5. 2025Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports
  6. 2026MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents
  7. 2026IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents
  8. 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome