mlsanityData to 7 Oct 2026

Counterfactual Effect Decomposition in Multi-Agent Sequential Decision Making

arXiv 2410.12539 ↗First posted 16 Oct 2024cs.AI cs.MAcited by 4

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

We address the challenge of explaining counterfactual outcomes in multi-agent Markov decision processes. In particular, we aim to explain the total counterfactual effect of an agent's action on the outcome of a realized scenario through its influence on the environment dynamics and the agents' behavior. To achieve this, we introduce a novel causal explanation formula that decomposes the counterfactual effect by attributing to each agent and state variable a score reflecting their respective contributions to the effect. First, we show that the total counterfactual effect of an agent's action can be decomposed into two components: one measuring the effect that propagates through all subsequent agents' actions and another related to the effect that propagates through the state transitions. Building on recent advancements in causal contribution analysis, we further decompose these two effects as follows. For the former, we consider agent-specific effects -- a causal concept that quantifies the counterfactual effect of an agent's action that propagates through a subset of agents. Based on this notion, we use Shapley value to attribute the effect to individual agents. For the latter, we consider the concept of structure-preserving interventions and attribute the effect to state variables based on their "intrinsic" contributions. Through extensive experimentation, we demonstrate the interpretability of our approach in a Gridworld environment with LLM-assisted agents and a sepsis management simulator.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 22 of its 53 references that are in the map. First posted on arXiv 16 Oct 2024.

Builds on

22 of 53 references.

References · 22

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  3. 2017Counterfactual Fairnesscited by 802
  4. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 279
  5. 2013Direct and Indirect Effectscited by 276
  6. 2018Path-Specific Counterfactual Fairnesscited by 179
  7. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  8. 2019Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Modelscited by 114
  9. 2020Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Modelscited by 107
  10. 2021Partial Counterfactual Identification from Observational and Experimental Datacited by 64
  11. 2023Collaborating with language models for embodied reasoningcited by 61
  12. 2020Shapley Flow: A Graph-based Approach to Interpreting Model Predictionscited by 51
10 more
  1. 2021Nested Counterfactual Identification from Arbitrary Surrogate Experimentscited by 38
  2. 2021Counterfactual Explanations in Sequential Decision Making Under Uncertaintycited by 36
  3. 2018Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibilitycited by 32
  4. 2022Counterfactual harmcited by 29
  5. 2021Achieving Counterfactual Fairness for Causal Banditcited by 15
  6. 2022A Causal Analysis of Harmcited by 15
  7. 2020Quantifying intrinsic causal contributions via structure preserving interventionscited by 12
  8. 2023Finding Counterfactually Optimal Action Sequences in Continuous State Spacescited by 12
  9. 2019Blameworthiness in Multi-Agent Settingscited by 11
  10. 2022Actual Causality and Responsibility Attribution in Decentralized Partially Observable Markov Decision Processescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 5
  2. 2026COSAC: Counterfactual Credit Assignment in Sequential Cooperative Teamscited by 1
  3. 2025Adaptive Accountability in Networked MAS: Tracing and Mitigating Emergent Norms at Scalecited by 0
  4. 2026Reach or Solve? Attributing Agentic RL Gains with Checkpoint Handoffscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Causal Explanations for Sequential Decision Making Under Uncertainty
  2. 2026Causal Attribution for Agentic Decisions: Estimators, Coupling, and a Traceability Specification
  3. 2026An Introduction to Causal Reinforcement Learning
  4. 2019Robust Multi-agent Counterfactual Prediction
  5. 2026Counterfactual Shapley Credit Assignment
  6. 2026Exact Is Easier: Credit Assignment for Cooperative LLM Agents
  7. 2025An Identifiable Cost-Aware Causal Decision-Making Framework Using Counterfactual Reasoning
  8. 2026Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures