mlsanityData to 7 Oct 2026

SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning

arXiv 2212.07489 ↗First posted 14 Dec 2022cs.LG cs.MAcited by 118

Not on the map. Why?

Abstract (arXiv)

The availability of challenging benchmarks has played a key role in the recent progress of machine learning. In cooperative multi-agent reinforcement learning, the StarCraft Multi-Agent Challenge (SMAC) has become a popular testbed for centralised training with decentralised execution. However, after years of sustained improvement on SMAC, algorithms now achieve near-perfect performance. In this work, we conduct new analysis demonstrating that SMAC lacks the stochasticity and partial observability to require complex *closed-loop* policies. In particular, we show that an *open-loop* policy conditioned only on the timestep can achieve non-trivial win rates for many SMAC scenarios. To address this limitation, we introduce SMACv2, a new version of the benchmark where scenarios are procedurally generated and require agents to generalise to previously unseen settings (from the same distribution) during evaluation. We also introduce the extended partial observability challenge (EPO), which augments SMACv2 to ensure meaningful partial observability. We show that these changes ensure the benchmark requires the use of *closed-loop* policies. We evaluate state-of-the-art algorithms on SMACv2 and show that it presents significant challenges not present in the original benchmark. Our analysis illustrates that SMACv2 addresses the discovered deficiencies of SMAC and can help benchmark the next generation of MARL methods. Videos of training are available at https://sites.google.com/view/smacv2.

Key facts. Cited by 118 papers in the map (data through 7 Oct 2026). Builds on 46 of its 58 references that are in the map. First posted on arXiv 14 Dec 2022.

Builds on

46 of 58 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  5. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  6. 2022A Generalist Agentcited by 802
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  8. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  9. 2019The StarCraft Multi-Agent Challengecited by 589
  10. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  11. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  12. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
34 more
  1. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  2. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
  3. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  4. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  5. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
  6. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  7. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 225
  8. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  9. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
  10. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  11. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  12. 2020The NetHack Learning Environmentcited by 196
  13. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 194
  14. 2019MAVEN: Multi-Agent Variational Explorationcited by 181
  15. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 180
  16. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  17. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  18. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  19. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  20. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  21. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
  22. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  23. 2019Increasing Generality in Machine Learning through Procedural Content Generationcited by 58
  24. 2019Obstacle Tower: A Generalization Challenge in Vision, Control, and Planningcited by 54
  25. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  26. 2019Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agentscited by 42
  27. 2020Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learningcited by 38
  28. 2020UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learningcited by 32
  29. 2018Pommerman: A Multi-Agent Playgroundcited by 30
  30. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  31. 2022You May Not Need Ratio Clipping in PPOcited by 19
  32. 2022Generalization in Cooperative Multi-Agent Systemscited by 18
  33. 2016TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Gamescited by 12
  34. 2022Trust Region Bounds for Decentralized PPO Under Non-stationaritycited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 19; 2024: 24; 2025: 39; 2026: 36.

118 papers.

Sort:

By area:

  1. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 80
  2. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 73
  3. 2024Simplifying Deep Temporal Difference Learningcited by 71
  4. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  5. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  6. 2024CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agentscited by 41
  7. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 31
  8. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 30
  9. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  10. 2025Large Language Models Miss the Multi-Agent Markcited by 25
  11. 2024Can VLMs Play Action Role-Playing Games? Take Black Myth Wukong as a Study Casecited by 23
  12. 2024Mixture of Experts in a Mixture of RL settingscited by 21
38 more
  1. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 20
  2. 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 19
  3. 2023Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?cited by 18
  4. 2025OvercookedV2: Rethinking Overcooked for Zero-Shot Coordinationcited by 14
  5. 2025CombatVLA: An Efficient Vision-Language-Action Model for Combat Tasks in 3D Action Role-Playing Gamescited by 13
  6. 2023Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learningcited by 12
  7. 2023From Explicit Communication to Tacit Cooperation:A Novel Paradigm for Cooperative MARLcited by 11
  8. 2024Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Modelscited by 10
  9. 2023IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARLcited by 9
  10. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  11. 2024ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularizationcited by 8
  12. 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learningcited by 8
  13. 2023Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learningcited by 7
  14. 2023RaidEnv: Exploring New Challenges in Automated Content Balancing for Boss Raid Gamescited by 7
  15. 2024Variational Offline Multi-agent Skill Discoverycited by 6
  16. 2024IntersectionZoo: Eco-driving for Benchmarking Multi-Agent Contextual Reinforcement Learningcited by 6
  17. 2025Deep Meta Coordination Graphs for Multi-agent Reinforcement Learningcited by 6
  18. 2025Adaptive Context Length Optimization with Low-Frequency Truncation for Multi-Agent Reinforcement Learningcited by 6
  19. 2024CoDreamer: Communication-Based Decentralised World Modelscited by 5
  20. 2024SMAC-Hard: Enabling Mixed Opponent Strategy Script and Self-play on SMACcited by 5
  21. 2025AVA: Attentive VLM Agent for Mastering StarCraft IIcited by 5
  22. 2025MALinZero: Efficient Low-Dimensional Search for Mastering Complex Multi-Agent Planningcited by 5
  23. 2024Multi-Agent Diagnostics for Robustness via Illuminated Diversitycited by 4
  24. 2024Visual Grounding for Object-Level Generalization in Reinforcement Learningcited by 4
  25. 2024Putting Data at the Centre of Offline Multi-Agent Reinforcement Learningcited by 4
  26. 2024Harnessing Language for Coordination: A Framework and Benchmark for LLM-Driven Multi-Agent Controlcited by 4
  27. 2024Novelty-Guided Data Reuse for Efficient and Diversified Multi-Agent Reinforcement Learningcited by 4
  28. 2025Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibrationcited by 4
  29. 2026NonZero: Interaction-Guided Exploration for Multi-Agent Monte Carlo Tree Searchcited by 4
  30. 2024QTypeMix: Enhancing Multi-Agent Cooperative Strategies through Heterogeneous and Homogeneous Value Decompositioncited by 3
  31. 2025VolleyBots: A Testbed for Multi-Drone Volleyball Game Combining Motion Control and Strategic Playcited by 3
  32. 2025MisoDICE: Multi-Agent Imitation from Unlabeled Mixed-Quality Demonstrationscited by 3
  33. 2025Skill-Aligned Fairness in Multi-Agent Learning for Collaboration in Healthcarecited by 3
  34. 2026GlobeDiff: State Diffusion Process for Partial Observability in Multi-Agent Systemscited by 3
  35. 2026Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learningcited by 3
  36. 2023Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Competitive Gamescited by 2
  37. 2023Inverse Factorized Q-Learning for Cooperative Multi-agent Imitation Learningcited by 2
  38. 2024Subequivariant Reinforcement Learning in 3D Multi-Entity Physical Environmentscited by 2

50 most cited of 118 All 118 (JSON)