# Efficient Reinforcement Learning for StarCraft by Abstract Forward Models and Transfer Learning

arXiv: https://arxiv.org/abs/1903.00715 · HTML: https://mlsanity.com/paper/1903.00715/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2019.

## Built on by (most cited first, 11 of 11)

- [Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach](https://mlsanity.com/paper/2312.11865/) (2023)
- [SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language models](https://mlsanity.com/paper/2401.17749/) (2024)
- [On Efficient Reinforcement Learning for Full-length Game of StarCraft II](https://mlsanity.com/paper/2209.11553/) (2022)
- [SMAC-R1: The Emergence of Intelligence in Decision-Making Tasks](https://mlsanity.com/paper/2410.16024/) (2024)
- [Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning towards Asynchronous Commercial Games](https://mlsanity.com/paper/2209.10055/) (2022)
- [EvoCurr: Self-evolving Curriculum with Behavior Code Generation for Complex Decision-making](https://mlsanity.com/paper/2508.09586/) (2025)
- [Adaptive Command: Real-Time Policy Adjustment via Language Models in StarCraft II](https://mlsanity.com/paper/2508.16580/) (2025)
- [Reinforcement Learning With Sparse-Executing Actions via Sparsity Regularization](https://mlsanity.com/paper/2105.08666/) (2021)
- [Rethinking of AlphaStar](https://mlsanity.com/paper/2108.03452/) (2021)
- [Enabling Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation](https://mlsanity.com/paper/2402.08184/) (2024)
- [Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search](https://mlsanity.com/paper/2410.11642/) (2024)

## Abstract (arXiv)

Injecting human knowledge is an effective way to accelerate reinforcement learning (RL). However, these methods are underexplored. This paper presents our discovery that an abstract forward model (thought-game (TG)) combined with transfer learning (TL) is an effective way. We take StarCraft II as our study environment. With the help of a designed TG, the agent can learn a 99% win-rate on a 64x64 map against the Level-7 built-in AI, using only 1.08 hours in a single commercial machine. We also show that the TG method is not as restrictive as it was thought to be. It can work with roughly designed TGs, and can also be useful when the environment changes. Comparing with previous model-based RL, we show TG is more effective. We also present a TG hypothesis that gives the influence of different fidelity levels of TG. For real games that have unequal state and action spaces, we proposed a novel XfrNet of which usefulness is validated while achieving a 90% win-rate against the cheating Level-10 AI. We argue that the TG method might shed light on further studies of efficient RL with human knowledge.