# Gym-μRTS: Toward Affordable Full Game Real-time Strategy Games Research with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2105.13807 · HTML: https://mlsanity.com/paper/2105.13807/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 21 May 2021.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [Action Space Shaping in Deep Reinforcement Learning](https://mlsanity.com/paper/2004.00980/) (2020)
- [TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game](https://mlsanity.com/paper/2011.13729/) (2020)
- [ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games](https://mlsanity.com/paper/1707.01067/) (2017)
- [TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game](https://mlsanity.com/paper/1809.07193/) (2018)
- [Modular Architecture for StarCraft II with Deep Reinforcement Learning](https://mlsanity.com/paper/1811.03555/) (2018)
- [Combinatorial Multi-armed Bandits for Real-Time Strategy Games](https://mlsanity.com/paper/1710.04805/) (2017)
- [Deep RTS: A Game Environment for Deep Reinforcement Learning in Real-Time Strategy Games](https://mlsanity.com/paper/1808.05032/) (2018)

## Built on by (most cited first, 19 of 19)

- [GPT for Games: An Updated Scoping Review (2020-2024)](https://mlsanity.com/paper/2411.00308/) (2024)
- [RaidEnv: Exploring New Challenges in Automated Content Balancing for Boss Raid Games](https://mlsanity.com/paper/2307.01676/) (2023)
- [No Prior Mask: Eliminate Redundant Action for Deep Reinforcement Learning](https://mlsanity.com/paper/2312.06258/) (2023)
- [Maze Learning using a Hyperdimensional Predictive Processing Cognitive Architecture](https://mlsanity.com/paper/2204.00619/) (2022)
- [TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents](https://mlsanity.com/paper/2601.05899/) (2026)
- [CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms](https://mlsanity.com/paper/2111.08819/) (2021)
- [ADAPTER-RL: Adaptation of Any Agent using Reinforcement Learning](https://mlsanity.com/paper/2311.11537/) (2023)
- [A Competition Winning Deep Reinforcement Learning Agent in microRTS](https://mlsanity.com/paper/2402.08112/) (2024)
- [PyTAG: Tabletop Games for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2405.18123/) (2024)
- [Strategy-Augmented Planning for Large Language Models via Opponent Exploitation](https://mlsanity.com/paper/2505.08459/) (2025)
- [InnateCoder: Learning Programmatic Options with Foundation Models](https://mlsanity.com/paper/2505.12508/) (2025)
- [Transformers as Policies for Variable Action Environments](https://mlsanity.com/paper/2301.03679/) (2023)
- [Neuro-symbolic Action Masking for Deep Reinforcement Learning](https://mlsanity.com/paper/2602.10598/) (2026)
- [Joint action loss for proximal policy optimization](https://mlsanity.com/paper/2301.10919/) (2023)
- [Centralized control for multi-agent RL in a complex Real-Time-Strategy game](https://mlsanity.com/paper/2304.13004/) (2023)
- [Reducing Action Space for Deep Reinforcement Learning via Causal Effect Estimation](https://mlsanity.com/paper/2501.14543/) (2025)
- [Play Style Identification Using Low-Level Representations of Play Traces in MicroRTS](https://mlsanity.com/paper/2507.10172/) (2025)
- [Empowering LLMs with Parameterized Skills for Adversarial Long-Horizon Planning](https://mlsanity.com/paper/2509.13127/) (2025)
- [Constrained Command-Conditioned Reinforcement Learning with Bandit Strategy Selection in Real-Time Strategy Games](https://mlsanity.com/paper/2610.11663/) (2026)

## Abstract (arXiv)

In recent years, researchers have achieved great success in applying Deep Reinforcement Learning (DRL) algorithms to Real-time Strategy (RTS) games, creating strong autonomous agents that could defeat professional players in StarCraft~II. However, existing approaches to tackle full games have high computational costs, usually requiring the use of thousands of GPUs and CPUs for weeks. This paper has two main contributions to address this issue: 1) We introduce Gym-μRTS (pronounced "gym-micro-RTS") as a fast-to-run RL environment for full-game RTS research and 2) we present a collection of techniques to scale DRL to play full-game μRTS as well as ablation studies to demonstrate their empirical importance. Our best-trained bot can defeat every μRTS bot we tested from the past μRTS competitions when working in a single-map setting, resulting in a state-of-the-art DRL agent while only taking about 60 hours of training using a single machine (one GPU, three vCPU, 16GB RAM). See the blog post at https://wandb.ai/vwxyzjn/gym-microrts-paper/reports/Gym-RTS-Toward-Affordable-Deep-Reinforcement-Learning-Research-in-Real-Time-Strategy-Games--Vmlldzo2MDIzMTg and the source code at https://github.com/vwxyzjn/gym-microrts-paper