# On Blame Attribution for Accountable Multi-Agent Sequential Decision Making

arXiv: https://arxiv.org/abs/2107.11927 · HTML: https://mlsanity.com/paper/2107.11927/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 64 references that are in the map.
- First posted on arXiv 26 Jul 2021.

## Builds on (most cited first, 16 of 16)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Towards Efficient Data Valuation Based on the Shapley Value](https://mlsanity.com/paper/1902.10275/) (2019)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1911.06854/) (2019)
- [Policy Certificates: Towards Accountable Reinforcement Learning](https://mlsanity.com/paper/1811.03056/) (2018)
- [A Marketplace for Data: An Algorithmic Solution](https://mlsanity.com/paper/1805.08125/) (2018)
- [Shapley Q-value: A Local Reward Approach to Solve Global Reward Games](https://mlsanity.com/paper/1907.05707/) (2019)
- [Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibility](https://mlsanity.com/paper/1810.05903/) (2018)
- [Human Perceptions on Moral Responsibility of AI: A Case Study in AI-Assisted Bail Decision-Making](https://mlsanity.com/paper/2102.00625/) (2021)
- [A Game-Theoretic Account of Responsibility Allocation](https://mlsanity.com/paper/2105.09129/) (2021)
- [Blameworthiness in Multi-Agent Settings](https://mlsanity.com/paper/1903.04102/) (2019)
- [Statistical Cost Sharing](https://mlsanity.com/paper/1703.03111/) (2017)
- [Learning Cooperative Games](https://mlsanity.com/paper/1505.00039/) (2015)
- [Program Actions as Actual Causes: A Building Block for Accountability](https://mlsanity.com/paper/1505.01131/) (2015)

## Built on by (most cited first, 13 of 13)

- [Performative Reinforcement Learning](https://mlsanity.com/paper/2207.00046/) (2022)
- [Traceability and Accountability in Role-Specialized Multi-Agent LLM Pipelines](https://mlsanity.com/paper/2510.07614/) (2025)
- [Actual Causality and Responsibility Attribution in Decentralized Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2204.00302/) (2022)
- [On Corruption-Robustness in Performative Reinforcement Learning](https://mlsanity.com/paper/2505.05609/) (2025)
- [Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives](https://mlsanity.com/paper/2506.09656/) (2025)
- [Towards Computationally Efficient Responsibility Attribution in Decentralized Partially Observable MDPs](https://mlsanity.com/paper/2302.12676/) (2023)
- [Analyzing Intentional Behavior in Autonomous Agents under Uncertainty](https://mlsanity.com/paper/2307.01532/) (2023)
- [Game-theoretic Counterfactual Explanation for Graph Neural Networks](https://mlsanity.com/paper/2402.06030/) (2024)
- [Do No Harm: A Counterfactual Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/2405.11669/) (2024)
- [Who Gets Credit or Blame? Attributing Accountability in Modern AI Systems](https://mlsanity.com/paper/2506.00175/) (2025)
- [Agent-Specific Effects: A Causal Effect Propagation Analysis in Multi-Agent MDPs](https://mlsanity.com/paper/2310.11334/) (2023)
- [Attribution-based Explanations for Markov Decision Processes](https://mlsanity.com/paper/2605.09780/) (2026)
- [AUDITA: certified auditing and causal attribution of adverse outcomes in autonomous multi-agent systems](https://mlsanity.com/paper/2608.22160/) (2026)

## Abstract (arXiv)

Blame attribution is one of the key aspects of accountable decision making, as it provides means to quantify the responsibility of an agent for a decision making outcome. In this paper, we study blame attribution in the context of cooperative multi-agent sequential decision making. As a particular setting of interest, we focus on cooperative decision making formalized by Multi-Agent Markov Decision Processes (MMDPs), and we analyze different blame attribution methods derived from or inspired by existing concepts in cooperative game theory. We formalize desirable properties of blame attribution in the setting of interest, and we analyze the relationship between these properties and the studied blame attribution methods. Interestingly, we show that some of the well known blame attribution methods, such as Shapley value, are not performance-incentivizing, while others, such as Banzhaf index, may over-blame agents. To mitigate these value misalignment and fairness issues, we introduce a novel blame attribution method, unique in the set of properties it satisfies, which trade-offs explanatory power (by under-blaming agents) for the aforementioned properties. We further show how to account for uncertainty about agents' decision making policies, and we experimentally: a) validate the qualitative properties of the studied blame attribution methods, and b) analyze their robustness to uncertainty.