Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systems
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Failure attribution in multi-agent systems -- pinpointing the exact step where a decisive error occurs -- is a critical yet unsolved challenge. Current methods treat this as a pattern recognition task over long conversation logs, leading to critically low step-level accuracy (below 17%), which renders them impractical for debugging complex systems. Their core weakness is a fundamental inability to perform robust counterfactual reasoning: to determine if correcting a single action would have actually averted the task failure. To bridge this counterfactual inference gap, we introduce Abduct-Act-Predict (A2P) Scaffolding, a novel agent framework that transforms failure attribution from pattern recognition into a structured causal inference task. A2P explicitly guides a large language model through a formal three-step reasoning process within a single inference pass: (1) Abduction, to infer the hidden root causes behind an agent's actions; (2) Action, to define a minimal corrective intervention; and (3) Prediction, to simulate the subsequent trajectory and verify if the intervention resolves the failure. This structured approach leverages the holistic context of the entire conversation while imposing a rigorous causal logic on the model's analysis. Our extensive experiments on the Who&When benchmark demonstrate its efficacy. On the Algorithm-Generated dataset, A2P achieves 47.46% step-level accuracy, a 2.85× improvement over the 16.67% of the baseline. On the more complex Hand-Crafted dataset, it achieves 29.31% step accuracy, a 2.43× improvement over the baseline's 12.07%. By reframing the problem through a causal lens, A2P Scaffolding provides a robust, verifiable, and significantly more accurate solution for automated failure attribution. Ours code are released at https://github.com/ResearAI/A2P.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 43 references that are in the map. First posted on arXiv 12 Sep 2025.
Builds on
29 of 43 references.
Closest ancestors · 1
Same direction, newest first
Other references · 28
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2023Let's Verify Step by Stepcited by 3,789
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
16 more
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
- 2022Solving math word problems with process- and outcome-based feedbackcited by 901
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 598
- 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
- 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
- 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
- 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
- 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 158
- 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
- 2023Can Large Language Models Infer Causation from Correlation?cited by 146
- 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoningcited by 94
- 2024CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMscited by 20
Built on by
Citing papers per year: 2025: 4; 2026: 10.
14 papers.
Sort:
By area:
- 2025DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systemscited by 24
- 2026Rethinking Failure Attribution in Multi-Agent Systems: A Multi-Perspective Benchmark and Evaluationcited by 24
- 2026AutoFigure: Generating and Refining Publication-Ready Scientific Illustrationscited by 23
- 2025Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systemscited by 12
- 2025Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Codecited by 10
- 2025Agentic Services Computingcited by 6
- 2026Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representationcited by 6
- 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 6
- 2026Heterogeneous Debate Engine: Identity-Grounded Cognitive Architecture for Resilient LLM-Based Ethical Tutoringcited by 5
- 2026DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systemscited by 2
- 2026RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enoughcited by 1
- 2026ASCon: A Direction-Aware Reciprocal Agent--Step Contextualization Model for Failure Attribution in Multi-Agent Systemscited by 1
- 2026Know the Shape, Find the Fault: Topology-Conditioned Diagnosis of Multi-Agent LLM Failures7 Oct 2026
- 2026DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems4 Sep 2026
- 2026ASCon: A Direction-Aware Reciprocal Agent--Step Contextualization Model for Failure Attribution in Multi-Agent Systems11 Aug 2026
- 2026RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough5 Aug 2026
- 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems14 May 2026
- 2026Heterogeneous Debate Engine: Identity-Grounded Cognitive Architecture for Resilient LLM-Based Ethical Tutoring28 Mar 2026
- 2026Rethinking Failure Attribution in Multi-Agent Systems: A Multi-Perspective Benchmark and Evaluation26 Mar 2026
- 2026Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representation23 Mar 2026
- 2026AutoFigure: Generating and Refining Publication-Ready Scientific Illustrations3 Feb 2026
- 20264D-ARE: Bridging the Attribution Gap in LLM Agent Requirements Engineering8 Jan 2026
- 2025DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems7 Dec 2025
- 2025Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems30 Oct 2025
2 more
- 2025Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Code2 Oct 2025
- 2025Agentic Services Computing29 Sep 2025
- 2025DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systemscited by 24
- 2025Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systemscited by 12
- 2025Agentic Services Computingcited by 6
- 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 6
- 2026RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enoughcited by 1
- 2026ASCon: A Direction-Aware Reciprocal Agent--Step Contextualization Model for Failure Attribution in Multi-Agent Systemscited by 1
- 2026Know the Shape, Find the Fault: Topology-Conditioned Diagnosis of Multi-Agent LLM Failurescited by 0
- 2026Know the Shape, Find the Fault: Topology-Conditioned Diagnosis of Multi-Agent LLM Failures7 Oct 2026
- 2026ASCon: A Direction-Aware Reciprocal Agent--Step Contextualization Model for Failure Attribution in Multi-Agent Systems11 Aug 2026
- 2026RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough5 Aug 2026
- 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems14 May 2026
- 2025DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems7 Dec 2025
- 2025Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems30 Oct 2025
- 2025Agentic Services Computing29 Sep 2025
- 2026Rethinking Failure Attribution in Multi-Agent Systems: A Multi-Perspective Benchmark and Evaluationcited by 24
- 2025Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Codecited by 10
- 2026Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representationcited by 6
- 2026DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systemscited by 2
- 20264D-ARE: Bridging the Attribution Gap in LLM Agent Requirements Engineeringcited by 0
- 2026DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems4 Sep 2026
- 2026Rethinking Failure Attribution in Multi-Agent Systems: A Multi-Perspective Benchmark and Evaluation26 Mar 2026
- 2026Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representation23 Mar 2026
- 20264D-ARE: Bridging the Attribution Gap in LLM Agent Requirements Engineering8 Jan 2026
- 2025Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Code2 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2026From Flat Logs to Causal Graphs: Hierarchical Failure Attribution for LLM-based Multi-Agent Systems
- 2026Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems
- 2026FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search
- 2026VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems
- 2026Error-Propagation Modeling for Failure Attribution in LLM-Based Multi-Agent Systems
- 2026MASPrism: Lightweight Failure Attribution for Multi-Agent Systems Using Prefill-Stage Signals
- 2026CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
- 2026Finding Where the Buck Stops: An Automated Failure Attribution-Based Reflection Framework for Multi-Agent Collaboration