Can Failed Trajectories Reveal Where Agents Go Wrong? Agentic Failure Attribution Without Step-Level Labels
On the map: LLM agents and tool use → Software engineering coding agents also fits: Agent skill learning and evolution
Abstract (arXiv)
Failure attribution for LLM agents aims to identify decisive steps responsible for task failure, which is critical for debugging and improving agentic systems. Existing methods rely on costly LLM prompting, step-level error annotations, or verified successful trajectories that lack failure signals for assistance. In this paper, we study failure attribution from step-unlabeled failure trajectories, where each trajectory is known to have failed but its decisive step is unknown. We propose SOAP (Spectral scOring with Attention-guided Propagation), an effective framework requiring neither step-level training labels nor a separate success-only reference set. SOAP first constructs a spectral reference space from LLM representations and assigns each step a base error score according to its projection behavior. Importantly, later steps may exhibit stronger failure signals because they inherit an earlier mistake, causing independent scorers to favor downstream consequences over the decisive error. To address this downstream contamination, SOAP derives soft step dependencies from attention patterns and propagates downstream error evidence toward its likely upstream sources. Extensive experiments show that SOAP consistently improves attribution performance over competitive baselines.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 27 of its 29 references that are in the map. First posted on arXiv 9 Oct 2026.
Builds on
27 of 29 references.
Closest ancestors · 18
Same direction, newest first
- 2026Adaptive Influence Graphs for Failure Attribution in Multi-Agent Systems25 Aug
- 2026TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories6 Aug
- 2026Tracing Agentic Failure from the Flow of Success14 Jul
- 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?10 Jul
- 2026StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems2 Jun
- 2026FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search30 May
- 2026MASPrism: Lightweight Failure Attribution for Multi-Agent Systems Using Prefill-Stage Signals8 May
- 2026Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems24 Apr
- 2026Towards Self-Improving Error Diagnosis in Multi-Agent Systems19 Apr
- 2026From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems27 Feb
- 2026AgentRx: Diagnosing AI Agent Failures from Execution Trajectories2 Feb
- 2025Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attribution6 Oct
6 more
- 2025Where LLM Agents Fail and How They can Learn From Failures29 Sep
- 2025CORRECT: COndensed eRror RECognition via knowledge Transfer in multi-agent systems28 Sep
- 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?3 Sep
- 2025TRAIL: Trace Reasoning and Agentic Issue Localization13 May
- 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems30 Apr
- 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
Other references · 9
- 2025Qwen3 Technical Reportcited by 8,820
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 99
- 2026MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Serverscited by 53
- 2025GraphTracer: Graph-Guided Failure Tracing in LLM Agents for Robust Multi-Turn Deep Searchcited by 23
- 2025RAFFLES: Reasoning-based Attribution of Faults for LLM Systemscited by 21
- 2026Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failurescited by 10
- 2025Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstructioncited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DeFA: Dependency-Guided Failure Attribution for LLM Agents
- 2026Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems
- 2026ReCast: Attribution-Oriented Step Representation Learning for LLM-Based Agent Systems
- 2026Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systems
- 2026VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems
- 2026ASCon: A Direction-Aware Reciprocal Agent--Step Contextualization Model for Failure Attribution in Multi-Agent Systems
- 2026Rethinking Failure Attribution in Multi-Agent Systems: A Multi-Perspective Benchmark and Evaluation
- 2026EDGE: Error Dependency Graph-Guided Multi-Error Attribution in Multi-Agent LLM Systems