mlsanityData to 7 Oct 2026

The StarCraft Multi-Agent Challenge

arXiv 1902.04043 ↗First posted 11 Feb 2019cs.LG cs.MA stat.MLcited by 589

Not on the map. Why?

Cited by 48 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In the last few years, deep multi-agent reinforcement learning (RL) has become a highly active area of research. A particularly challenging class of problems in this area is partially observable, cooperative, multi-agent learning, in which teams of agents must learn to coordinate their behaviour while conditioning only on their private observations. This is an attractive research area since such problems are relevant to a large number of real-world systems and are also more amenable to evaluation than general-sum problems. Standardised environments such as the ALE and MuJoCo have allowed single-agent RL to move beyond toy domains, such as grid worlds. However, there is no comparable benchmark for cooperative multi-agent RL. As a result, most papers in this field use one-off toy problems, making it difficult to measure real progress. In this paper, we propose the StarCraft Multi-Agent Challenge (SMAC) as a benchmark problem to fill this gap. SMAC is based on the popular real-time strategy game StarCraft II and focuses on micromanagement challenges where each unit is controlled by an independent agent that must act based on local observations. We offer a diverse set of challenge maps and recommendations for best practices in benchmarking and evaluations. We also open-source a deep multi-agent RL learning framework including state-of-the-art algorithms. We believe that SMAC can provide a standard benchmark environment for years to come. Videos of our best agents for several SMAC scenarios are available at: https://youtu.be/VZ7zmQ_obZ0.

Key facts. Cited by 589 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 76; 2022: 95; 2023: 107; 2024: 104; 2025: 117; 2026: 90.

589 papers.

Sort:

By area:

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  3. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  4. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  5. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  6. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  7. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 129
  8. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  9. 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 114
  10. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 113
  11. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  12. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
38 more
  1. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 85
  2. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 83
  3. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 81
  4. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 80
  5. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 78
  6. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 73
  7. 2024Simplifying Deep Temporal Difference Learningcited by 71
  8. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  9. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  10. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  11. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 63
  12. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 59
  13. 2023POPGym: Benchmarking Partially Observable Reinforcement Learningcited by 59
  14. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  15. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  16. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  17. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  18. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 57
  19. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  20. 2021Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learningcited by 54
  21. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  22. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  23. 2024COMBO: Compositional World Models for Embodied Multi-Agent Cooperationcited by 47
  24. 2024Learning Multi-Agent Communication from Graph Modeling Perspectivecited by 47
  25. 2025LLM Collaboration With Multi-Agent Reinforcement Learningcited by 47
  26. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  27. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 46
  28. 2021Settling the Variance of Multi-Agent Policy Gradientscited by 45
  29. 2023Welfare Diplomacy: Benchmarking Language Model Cooperationcited by 43
  30. 2023Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularizationcited by 42
  31. 2025Language Self-Play For Data-Free Trainingcited by 42
  32. 2024CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agentscited by 41
  33. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  34. 2021Shapley Counterfactual Credits for Multi-Agent Reinforcement Learningcited by 37
  35. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  36. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 36
  37. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  38. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35

50 most cited of 589 All 589 (JSON)