Agent-Specific Effects: A Causal Effect Propagation Analysis in Multi-Agent MDPs
Not on the map. Why?
Abstract (arXiv)
Establishing causal relationships between actions and outcomes is fundamental for accountable multi-agent decision-making. However, interpreting and quantifying agents' contributions to such relationships pose significant challenges. These challenges are particularly prominent in the context of multi-agent sequential decision-making, where the causal effect of an agent's action on the outcome depends on how other agents respond to that action. In this paper, our objective is to present a systematic approach for attributing the causal effects of agents' actions to the influence they exert on other agents. Focusing on multi-agent Markov decision processes, we introduce agent-specific effects (ASE), a novel causal quantity that measures the effect of an agent's action on the outcome that propagates through other agents. We then turn to the counterfactual counterpart of ASE (cf-ASE), provide a sufficient set of conditions for identifying cf-ASE, and propose a practical sampling-based algorithm for estimating it. Finally, we experimentally evaluate the utility of cf-ASE through a simulation-based testbed, which includes a sepsis management environment.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 54 references that are in the map. First posted on arXiv 17 Oct 2023.
Builds on
24 of 54 references.
References · 24
- 2013Direct and Indirect Effectscited by 280
- 2018Path-Specific Counterfactual Fairnesscited by 181
- 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
- 2019Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Modelscited by 117
- 2020Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Modelscited by 107
- 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 103
- 2018Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Searchcited by 82
- 2020Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentationcited by 59
- 2021Agent Incentives: A Causal Perspectivecited by 51
- 2023Counterfactual Identifiability of Bijective Causal Modelscited by 43
- 2023Characterizing Manipulation from AI Systemscited by 43
- 2019Solving Discounted Stochastic Two-Player Games with Near-Optimal Time and Sample Complexitycited by 42
12 more
- 2021Nested Counterfactual Identification from Arbitrary Surrogate Experimentscited by 38
- 2018Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibilitycited by 32
- 2022Counterfactual harmcited by 29
- 2019Feature-Based Q-Learning for Two-Player Stochastic Gamescited by 24
- 2017Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaborationcited by 21
- 2022Path-Specific Objectives for Safer Agent Incentivescited by 20
- 2019Learning to Collaborate in Markov Decision Processescited by 14
- 2021On Blame Attribution for Accountable Multi-Agent Sequential Decision Makingcited by 13
- 2023Finding Counterfactually Optimal Action Sequences in Continuous State Spacescited by 13
- 2022Actual Causality and Responsibility Attribution in Decentralized Partially Observable Markov Decision Processescited by 8
- 2021Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Interventioncited by 5
- 2023Towards Computationally Efficient Responsibility Attribution in Decentralized Partially Observable MDPscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.