How much can change in a year? Revisiting Evaluation in Multi-Agent Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Establishing sound experimental standards and rigour is important in any growing field of research. Deep Multi-Agent Reinforcement Learning (MARL) is one such nascent field. Although exciting progress has been made, MARL has recently come under scrutiny for replicability issues and a lack of standardised evaluation methodology, specifically in the cooperative setting. Although protocols have been proposed to help alleviate the issue, it remains important to actively monitor the health of the field. In this work, we extend the database of evaluation methodology previously published by containing meta-data on MARL publications from top-rated conferences and compare the findings extracted from this updated database to the trends identified in their work. Our analysis shows that many of the worrying trends in performance reporting remain. This includes the omission of uncertainty quantification, not reporting all relevant evaluation details and a narrowing of algorithmic development classes. Promisingly, we do observe a trend towards more difficult scenarios in SMAC-v1, which if continued into SMAC-v2 will encourage novel algorithmic development. Our data indicate that replicability needs to be approached more proactively by the MARL community to ensure trust in the field as we move towards exciting new frontiers.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 13 Dec 2023.
Builds on
35 of 44 references.
References · 35
- 2017Proximal Policy Optimization Algorithmscited by 14,592
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,223
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,525
- 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 910
- 2017Counterfactual Multi-Agent Policy Gradientscited by 739
- 2017Deep Reinforcement Learning that Matterscited by 734
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 700
- 2019The StarCraft Multi-Agent Challengecited by 592
- 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 493
- 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 479
- 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 379
- 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 268
23 more
- 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
- 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
- 2020The NetHack Learning Environmentcited by 197
- 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 185
- 2019MAVEN: Multi-Agent Variational Explorationcited by 183
- 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
- 2020Explainable Reinforcement Learning: A Surveycited by 139
- 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
- 2020Explainability in Deep Reinforcement Learningcited by 96
- 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
- 2018Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networkscited by 70
- 2021A Survey on Interpretable Reinforcement Learningcited by 70
- 2018Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 68
- 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
- 2018How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experimentscited by 59
- 2019Obstacle Tower: A Generalization Challenge in Vision, Control, and Planningcited by 54
- 2019Simplified Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 48
- 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
- 2019Reasoning about Hypothetical Agent Behaviours and their Parameterscited by 24
- 2020Multi-agent Reinforcement Learning for Resource Allocation in IoT networks with Edge Computingcited by 13
- 2019Autonomous Air Traffic Controller: A Deep Multi-Agent Reinforcement Learning Approachcited by 10
- 2020A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learningcited by 5
- 2021ShinRL: A Library for Evaluating RL Algorithms from Theoretical and Practical Perspectivescited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.