mlsanityData to 14 Oct 2026

Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systems

arXiv 2509.10401 ↗First posted 12 Sep 2025cs.AI cs.CLcited by 14

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Failure attribution in multi-agent systems -- pinpointing the exact step where a decisive error occurs -- is a critical yet unsolved challenge. Current methods treat this as a pattern recognition task over long conversation logs, leading to critically low step-level accuracy (below 17%), which renders them impractical for debugging complex systems. Their core weakness is a fundamental inability to perform robust counterfactual reasoning: to determine if correcting a single action would have actually averted the task failure. To bridge this counterfactual inference gap, we introduce Abduct-Act-Predict (A2P) Scaffolding, a novel agent framework that transforms failure attribution from pattern recognition into a structured causal inference task. A2P explicitly guides a large language model through a formal three-step reasoning process within a single inference pass: (1) Abduction, to infer the hidden root causes behind an agent's actions; (2) Action, to define a minimal corrective intervention; and (3) Prediction, to simulate the subsequent trajectory and verify if the intervention resolves the failure. This structured approach leverages the holistic context of the entire conversation while imposing a rigorous causal logic on the model's analysis. Our extensive experiments on the Who&When benchmark demonstrate its efficacy. On the Algorithm-Generated dataset, A2P achieves 47.46% step-level accuracy, a 2.85× improvement over the 16.67% of the baseline. On the more complex Hand-Crafted dataset, it achieves 29.31% step accuracy, a 2.43× improvement over the baseline's 12.07%. By reframing the problem through a causal lens, A2P Scaffolding provides a robust, verifiable, and significantly more accurate solution for automated failure attribution. Ours code are released at https://github.com/ResearAI/A2P.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 43 references that are in the map. First posted on arXiv 12 Sep 2025.

Builds on

29 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems30 Apr

Other references · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  10. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  12. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
16 more
  1. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  2. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  3. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  4. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  5. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  6. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  7. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 598
  8. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  9. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
  10. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  11. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
  12. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 158
  13. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
  14. 2023Can Large Language Models Infer Causation from Correlation?cited by 146
  15. 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoningcited by 94
  16. 2024CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 10.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems
  2. 2026Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems
  3. 2026FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search
  4. 2026VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems
  5. 2026Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems
  6. 2026MASPrism: Lightweight Failure Attribution for Multi-Agent Systems Using Prefill-Stage Signals
  7. 2026CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
  8. 2026Finding Where the Buck Stops: An Automated Failure Attribution-Based Reflection Framework for Multi-Agent Collaboration