mlsanityData to 14 Oct 2026

Multi-Turn Evaluation of Deep Research Agents Under Process-Level Feedback

arXiv 2606.09748 ↗First posted 8 Jun 2026cs.AI cs.CL cs.LGcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Existing benchmarks for deep research agents (DRAs) assess only single-shot outputs, ignoring a key question: can DRAs improve their reports when guided by feedback? To investigate this, we conduct a multi-turn evaluation of DRAs under two feedback settings: self-reflection, in which the agent revises its report without any external diagnostic signal, and process-level feedback, in which the agent receives guidance targeting gaps in its research strategy. To enable process-level feedback, we design Research Gap Inference (RGI), a method that analyzes patterns of satisfied and unsatisfied rubric criteria to infer research-process gaps. Our analysis reveals three key findings: (i) under self-reflection, agents incorporate and regress on rubric criteria at nearly equal rates, yielding negligible net improvement; (ii) a single round of process-level feedback yields substantial gains, raising the normalized score by approximately 8-15 points and yielding a roughly 35-40% incorporation rate; (iii) these gains do not compound over subsequent turns, as agents regress on up to 24% of previously satisfied criteria when rewriting the full report to address remaining gaps. Even with targeted guidance, reliable multi-turn improvement remains out of reach for the DRA architectures we evaluate. Our code and results are publicly available at https://github.com/sabharwalrishabh/Multi-Turn-Evaluation-of-DRAs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 17 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

12 of 17 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision19 Jan
  2. 2026DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports13 Jan
  3. 2026IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents10 Jan
  4. 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents10 Nov
  5. 2025LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild16 Oct
  6. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents13 Jun

Other references · 6

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  3. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  4. 2025ResearcherBench: Evaluating Deep AI Research Systems on the Frontiers of Scientific Inquirycited by 32
  5. 2025DRBench: A Realistic Benchmark for Enterprise Deep Researchcited by 22
  6. 2026DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivitycited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports
  2. 2025ADRA-Bank: A Modular Benchmark for Academic Deep Research Agents
  3. 2026Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
  4. 2025How Far Are We from Genuinely Useful Deep Research Agents?
  5. 2026DR3-Eval: Towards Realistic and Reproducible Deep Research Evaluation
  6. 2026Evaluating Stochasticity in Deep Research Agents
  7. 2026TRACE: Trajectory-Aware Comprehensive Evaluation for Deep Research Agents
  8. 2026DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoning