mlsanityData to 7 Oct 2026

FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search

arXiv 2606.00765 ↗First posted 30 May 2026cs.AIcited by 9

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

LLM-based agents increasingly solve complex tasks through long trajectories involving reasoning steps, tool calls, and inter-agent communication. However, when these agents fail, it is often unclear which agent caused the failure and which step introduced the decisive error. This attribution problem is challenging because mistakes can propagate across the trajectory: later actions may appear incorrect, but only because they depend on an earlier corrupted state. Therefore, failure attribution cannot be treated as independent step-level classification. We propose FALAT, a diagnostic framework for failure attribution in LLM agent trajectories. FALAT frames attribution as a dependency-guided search problem. It first constructs an expectation of how the task should be solved and uses this expectation to identify suspicious regions in the trajectory. It then traces dependencies among decisions, tool outputs, and agent messages to distinguish error-introducing steps from steps that merely inherit or propagate prior mistakes. Finally, FALAT evaluates whether correcting a candidate step would be sufficient to recover the expected outcome, allowing it to identify both the responsible agent and the decisive failure step. We evaluate FALAT on the Who&When benchmark, which includes both algorithm-generated and hand-crafted multi-agent failure trajectories. The results show that FALAT consistently improves responsible-agent and decisive-step attribution. Its best configurations achieve 46.0% step-level accuracy on algorithm-generated trajectories and 29.1% on the more challenging hand-crafted trajectories, outperforming specialized attribution baselines and direct prompting with standalone LLMs. These findings suggest that dependency-aware reasoning is essential for reliable failure diagnosis in LLM agent systems.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 33 of its 39 references that are in the map. First posted on arXiv 30 May 2026.

Builds on

33 of 39 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories23 Jun
  2. 2025Guided Search Strategies in Non-Serializable Environments with Applications to Software Engineering Agents19 May
  3. 2024AFlow: Automating Agentic Workflow Generation14 Oct

Other references · 30

  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  5. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  6. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  7. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  8. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  9. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  10. 2024Automated Design of Agentic Systemscited by 360
  11. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  12. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
18 more
  1. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 170
  2. 2025Multi-agent Architecture Search via Agentic Supernetcited by 164
  3. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 155
  4. 2025Where LLM Agents Fail and How They can Learn From Failurescited by 118
  5. 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 105
  6. 2025MasRouter: Learning to Route LLMs for Multi-Agent Systemscited by 80
  7. 2025EvoFlow: Evolving Diverse Agentic Workflows On The Flycited by 49
  8. 2025TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Usecited by 46
  9. 2025UTBoost: Rigorous Evaluation of Coding Agents on SWE-Benchcited by 44
  10. 2024Adaptive In-conversation Team Building for Language Model Agentscited by 38
  11. 2025Optimizing Model Selection for Compound AI Systemscited by 34
  12. 2025Who is Introducing the Failure? Automatically Attributing Failures of Multi-Agent Systems via Spectrum Analysiscited by 33
  13. 2025Adaptive Graph Pruning for Multi-Agent Communicationcited by 28
  14. 2025Where Did It All Go Wrong? A Hierarchical Look into Multi-Agent Error Attributioncited by 25
  15. 2025GraphTracer: Graph-Guided Failure Tracing in LLM Agents for Robust Multi-Turn Deep Searchcited by 22
  16. 2026From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systemscited by 17
  17. 2025Agent-SAMA: State-Aware Mobile Assistantcited by 12
  18. 2024Order Matters! An Empirical Study on Large Language Models' Input Order Bias in Software Fault Localizationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026DeFA: Dependency-Guided Failure Attribution for LLM Agents
  2. 2026StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems
  3. 2026Redundant or Necessary? A Benchmark for Detecting Redundant Steps in Agent Trajectories
  4. 2026Tracing Agentic Failure from the Flow of Success
  5. 2026Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems
  6. 2026VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems
  7. 2026Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systems
  8. 2026TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories