mlsanityData to 7 Oct 2026

Efficiently Quantifying Individual Agent Importance in Cooperative MARL

arXiv 2312.08466 ↗First posted 13 Dec 2023cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Measuring the contribution of individual agents is challenging in cooperative multi-agent reinforcement learning (MARL). In cooperative MARL, team performance is typically inferred from a single shared global reward. Arguably, among the best current approaches to effectively measure individual agent contributions is to use Shapley values. However, calculating these values is expensive as the computational complexity grows exponentially with respect to the number of agents. In this paper, we adapt difference rewards into an efficient method for quantifying the contribution of individual agents, referred to as Agent Importance, offering a linear computational complexity relative to the number of agents. We show empirically that the computed values are strongly correlated with the true Shapley values, as well as the true underlying individual agent rewards, used as the ground truth in environments where these are available. We demonstrate how Agent Importance can be used to help study MARL systems by diagnosing algorithmic failures discovered in prior MARL benchmarking work. Our analysis illustrates Agent Importance as a valuable explainability component for future MARL benchmarks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 37 of its 54 references that are in the map. First posted on arXiv 13 Dec 2023.

Builds on

37 of 54 references.

References · 37

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  5. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  6. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  8. 2019The StarCraft Multi-Agent Challengecited by 589
  9. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  10. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  11. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
  12. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 194
25 more
  1. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  2. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
  3. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  4. 2020Explainable Reinforcement Learning: A Surveycited by 139
  5. 2019Behaviour Suite for Reinforcement Learningcited by 129
  6. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 114
  7. 2020Explainability in Deep Reinforcement Learningcited by 96
  8. 2020Qatten: A General Framework for Cooperative Multiagent Reinforcement Learningcited by 81
  9. 2019Shapley Q-value: A Local Reward Approach to Solve Global Reward Gamescited by 71
  10. 2018Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networkscited by 70
  11. 2021A Survey on Interpretable Reinforcement Learningcited by 70
  12. 2018Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 67
  13. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  14. 2019Simplified Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 47
  15. 2021SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learningcited by 26
  16. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  17. 2021Collective eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning with Shapley Valuescited by 24
  18. 2020Q-value Path Decomposition for Deep Multiagent Reinforcement Learningcited by 23
  19. 2018Where Did My Optimum Go?: An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methodscited by 17
  20. 2020Multi-agent Reinforcement Learning for Resource Allocation in IoT networks with Edge Computingcited by 13
  21. 2019AI for Explaining Decisions in Multi-Agent Environmentscited by 12
  22. 2022Stable and Efficient Shapley Value-Based Reward Reallocation for Multi-Agent Reinforcement Learning of Autonomous Vehiclescited by 11
  23. 2020A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learningcited by 5
  24. 2021ShinRL: A Library for Evaluating RL Algorithms from Theoretical and Practical Perspectivescited by 5
  25. 2023The challenge of redundancy on multi-agent value factorisationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024Explaining Decisions of Agents in Mixed-Motive Gamescited by 4
  2. 2025Understanding Action Effects through Instrumental Empowerment in Multi-Agent Reinforcement Learningcited by 0
  3. 2025Explaining Decentralized Multi-Agent Reinforcement Learning Policiescited by 0
  4. 2026Characterizing MARL for Energy Control: A Multi-KPI Benchmark on the CityLearn Environmentcited by 0