# StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learning

arXiv: https://arxiv.org/abs/1804.00810 · HTML: https://mlsanity.com/paper/1804.00810/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2018.

## Built on by (most cited first, 25 of 25)

- [UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers](https://mlsanity.com/paper/2101.08001/) (2021)
- [Bridging RL Theory and Practice with the Effective Horizon](https://mlsanity.com/paper/2304.09853/) (2023)
- [Transformers in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.05979/) (2023)
- [A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat](https://mlsanity.com/paper/2212.03830/) (2022)
- [A Versatile Multi-Agent Reinforcement Learning Benchmark for Inventory Management](https://mlsanity.com/paper/2306.07542/) (2023)
- [Multi-task Safe Reinforcement Learning for Navigating Intersections in Dense Traffic](https://mlsanity.com/paper/2202.09644/) (2022)
- [Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning](https://mlsanity.com/paper/2105.14780/) (2021)
- [Decision Making in Monopoly using a Hybrid Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2103.00683/) (2021)
- [Transfer Dynamics in Emergent Evolutionary Curricula](https://mlsanity.com/paper/2203.10941/) (2022)
- [Asynchronous Curriculum Experience Replay: A Deep Reinforcement Learning Approach for UAV Autonomous Motion Control in Unknown Dynamic Environments](https://mlsanity.com/paper/2207.01251/) (2022)
- [Similarity-based Knowledge Transfer for Cross-Domain Reinforcement Learning](https://mlsanity.com/paper/2312.03764/) (2023)
- [Hybrid Training for Enhanced Multi-task Generalization in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2408.13567/) (2024)
- [Teacher-student curriculum learning for reinforcement learning](https://mlsanity.com/paper/2210.17368/) (2022)
- [Improved Reinforcement Learning in Cooperative Multi-agent Environments Using Knowledge Transfer](https://mlsanity.com/paper/2107.09807/) (2021)
- [Forecasting Evolution of Clusters in Game Agents with Hebbian Learning](https://mlsanity.com/paper/2209.06904/) (2022)
- [Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery](https://mlsanity.com/paper/2311.14270/) (2023)
- [Enabling Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation](https://mlsanity.com/paper/2402.08184/) (2024)
- [Active Reinforcement Learning over MDPs](https://mlsanity.com/paper/2108.02323/) (2021)
- [All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization](https://mlsanity.com/paper/2310.00964/) (2023)
- [FM3Q: Factorized Multi-Agent MiniMax Q-Learning for Two-Team Zero-Sum Markov Game](https://mlsanity.com/paper/2402.00738/) (2024)
- [Unveiling Options with Neural Decomposition](https://mlsanity.com/paper/2410.11262/) (2024)
- [Transfer Reinforcement Learning in Heterogeneous Action Spaces using Subgoal Mapping](https://mlsanity.com/paper/2410.14484/) (2024)
- [NeuroPAL: Punctuated Anytime Learning with Neuroevolution for Macromanagement in Starcraft: Brood War](https://mlsanity.com/paper/2506.10384/) (2025)
- [Two-Bridge: Exclusive Objectives and Extended Horizon StarCraft II Benchmark](https://mlsanity.com/paper/2603.06608/) (2026)
- [Hierarchical Reinforcement Learning in StarCraft Micromanagement with Influence Maps and Cluster-based Scripts](https://mlsanity.com/paper/2606.30092/) (2026)

## Abstract (arXiv)

Real-time strategy games have been an important field of game artificial intelligence in recent years. This paper presents a reinforcement learning and curriculum transfer learning method to control multiple units in StarCraft micromanagement. We define an efficient state representation, which breaks down the complexity caused by the large state space in the game environment. Then a parameter sharing multi-agent gradientdescent Sarsa(λ) (PS-MAGDS) algorithm is proposed to train the units. The learning policy is shared among our units to encourage cooperative behaviors. We use a neural network as a function approximator to estimate the action-value function, and propose a reward function to help units balance their move and attack. In addition, a transfer learning method is used to extend our model to more difficult scenarios, which accelerates the training process and improves the learning performance. In small scale scenarios, our units successfully learn to combat and defeat the built-in AI with 100% win rates. In large scale scenarios, curriculum transfer learning method is used to progressively train a group of units, and shows superior performance over some baseline methods in target scenarios. With reinforcement learning and curriculum transfer learning, our units are able to learn appropriate strategies in StarCraft micromanagement scenarios.