Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Deep-research agents solve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis. Evaluation based on final answers shows whether an agent succeeds, but not which parts of the trajectory make the answer unreliable. We study span-level error localization for deep-research agents. We collect 2,790 real trajectories from two agent frameworks, three backbone models, and three benchmarks, convert raw logs into semantic spans, and annotate harmful error spans through LLM-assisted expert review. From these annotations, we build TELBench, a 1,000-instance benchmark for identifying error spans among normal exploration, failed searches, tentative hypotheses, and harmless noise. We further propose DRIFT, a claim-centric auditing framework that tracks agent claims, checks their support in trajectory evidence, and marks spans where unsupported or conflicting claims affect the answer path. Experiments across model families and auditing frameworks show that DRIFT improves span-level error localization and first-error accuracy by up to 30 percentage points. Our work provides a process-level view of reliability in deep-research agents.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 24 of its 47 references that are in the map. First posted on arXiv 1 Jun 2026.
Builds on
24 of 47 references.
Closest ancestors · 3
Same direction, newest first
- 2025LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild16 Oct
- 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents13 Jun
- 2025DeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research25 May
Other references · 21
- 2023Let's Verify Step by Stepcited by 3,598
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
- 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277
- 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 170
- 2023AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Webcited by 144
- 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoningcited by 134
- 2025xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluationscited by 118
- 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Modelscited by 98
9 more
- 2025TRAIL: Trace Reasoning and Agentic Issue Localizationcited by 98
- 2026AgentRx: Diagnosing AI Agent Failures from Execution Trajectoriescited by 53
- 2025Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?cited by 43
- 2026AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agentscited by 28
- 2026Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systemscited by 22
- 2025Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agentscited by 20
- 2025DRBench: A Realistic Benchmark for Enterprise Deep Researchcited by 19
- 2026CodeTracer: Towards Traceable Agent Statescited by 11
- 2026Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?cited by 4
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 6
- 2026Mint-Agent: Introducing Finance-Native Agentic Foundation Modelscited by 0
- 2026SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphscited by 0
- 2026AgentPProf: Semantic Profiler for Long Horizon AI Agentscited by 0
- 2026Auditing Agent Actions through Query-Conditioned Attributioncited by 0
- 2026Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Trainingcited by 0
- 2026Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training30 Sep 2026
- 2026Auditing Agent Actions through Query-Conditioned Attribution27 Sep 2026
- 2026AgentPProf: Semantic Profiler for Long Horizon AI Agents14 Sep 2026
- 2026SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs9 Sep 2026
- 2026Mint-Agent: Introducing Finance-Native Agentic Foundation Models17 Aug 2026
- 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap29 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Explore Before Committing: Hypothesis-Guided Search for Deep Research Agents
- 2026REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces
- 2026SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents
- 2026TRACE: Trajectory-Aware Comprehensive Evaluation for Deep Research Agents
- 2026TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories
- 2026Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
- 2026From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents
- 2026Who is the Agent to Blame? Localizing Faithfulness and Citation Mistakes in Agentic Deep Research