mlsanityData to 7 Oct 2026

StarCraft II: A New Challenge for Reinforcement Learning

arXiv 1708.04782 ↗First posted 16 Aug 2017cs.LG cs.AIcited by 268

Not on the map. Why?

Cited by 27 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper introduces SC2LE (StarCraft II Learning Environment), a reinforcement learning environment based on the StarCraft II game. This domain poses a new grand challenge for reinforcement learning, representing a more difficult class of problems than considered in most prior work. It is a multi-agent problem with multiple players interacting; there is imperfect information due to a partially observed map; it has a large action space involving the selection and control of hundreds of units; it has a large state space that must be observed solely from raw input feature planes; and it has delayed credit assignment requiring long-term strategies over thousands of steps. We describe the observation, action, and reward specification for the StarCraft II domain and provide an open source Python-based interface for communicating with the game engine. In addition to the main game maps, we provide a suite of mini-games focusing on different elements of StarCraft II gameplay. For the main game maps, we also provide an accompanying dataset of game replay data from human expert players. We give initial baseline results for neural networks trained from this data to predict game outcomes and player actions. Finally, we present initial baseline results for canonical deep reinforcement learning agents applied to the StarCraft II domain. On the mini-games, these agents learn to achieve a level of play that is comparable to a novice player. However, when trained on the main game, these agents are unable to make significant progress. Thus, SC2LE offers a new and challenging environment for exploring deep reinforcement learning algorithms and architectures.

Key facts. Cited by 268 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Aug 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 2; 2021: 60; 2022: 57; 2023: 44; 2024: 41; 2025: 44; 2026: 19.

268 papers.

Sort:

By area:

  1. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  2. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  3. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  4. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  5. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  6. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  7. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  8. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  9. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
  10. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 83
  11. 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approachcited by 79
  12. 2021A Survey on Interpretable Reinforcement Learningcited by 70
38 more
  1. 2022Full-Spectrum Out-of-Distribution Detectioncited by 63
  2. 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 56
  3. 2021Decentralized Q-Learning in Zero-sum Markov Gamescited by 53
  4. 2025LLM Collaboration With Multi-Agent Reinforcement Learningcited by 47
  5. 2024CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agentscited by 41
  6. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agentscited by 41
  7. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  8. 2021Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabicited by 37
  9. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 36
  10. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  11. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  12. 2023Interactive Natural Language Processingcited by 31
  13. 2022K-level Reasoning for Zero-Shot Coordination in Hanabicited by 30
  14. 2022Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangiancited by 29
  15. 2023Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Gamescited by 29
  16. 2021Learning Graphon Mean Field Games and Approximate Nash Equilibriacited by 28
  17. 2023Early Neuron Alignment in Two-layer ReLU Networks with Small Initializationcited by 28
  18. 2021Confidence Calibration for Domain Generalization under Covariate Shiftcited by 27
  19. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  20. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  21. 2025A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectivescited by 27
  22. 2024BattleAgentBench: A Benchmark for Evaluating Cooperation and Competition Capabilities of Language Models in Multi-Agent Systemscited by 26
  23. 2023The Generalization Gap in Offline Reinforcement Learningcited by 25
  24. 2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
  25. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  26. 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23
  27. 2022On the Limitations of Elo: Real-World Games, are Transitive, not Additivecited by 23
  28. 2023Maximum State Entropy Exploration using Predecessor and Successor Representationscited by 23
  29. 2021Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversitycited by 22
  30. 2024Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolfcited by 22
  31. 2025DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environmentscited by 22
  32. 2022CompoSuite: A Compositional Reinforcement Learning Benchmarkcited by 21
  33. 2021Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Spacecited by 20
  34. 2021Gym-μRTS: Toward Affordable Full Game Real-time Strategy Games Research with Deep Reinforcement Learningcited by 19
  35. 2021Explaining Bayesian Neural Networkscited by 18
  36. 2021Hyperparameter Selection for Imitation Learningcited by 17
  37. 2021Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learningcited by 17
  38. 2022Multi-Agent Path Finding with Prioritized Communication Learningcited by 17

50 most cited of 268 All 268 (JSON)