# TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game

arXiv: https://arxiv.org/abs/1809.07193 · HTML: https://mlsanity.com/paper/1809.07193/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2018.

## Built on by (most cited first, 14 of 14)

- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Gym-μRTS: Toward Affordable Full Game Real-time Strategy Games Research with Deep Reinforcement Learning](https://mlsanity.com/paper/2105.13807/) (2021)
- [On Efficient Reinforcement Learning for Full-length Game of StarCraft II](https://mlsanity.com/paper/2209.11553/) (2022)
- [An Introduction of mini-AlphaStar](https://mlsanity.com/paper/2104.06890/) (2021)
- [Toward Causal-Aware RL: State-Wise Action-Refined Temporal Difference](https://mlsanity.com/paper/2201.00354/) (2022)
- [On games and simulators as a platform for development of artificial intelligence for command and control](https://mlsanity.com/paper/2110.11305/) (2021)
- [Memory-Augmented State Machine Prompting: A Novel LLM Agent Framework for Real-Time Strategy Games](https://mlsanity.com/paper/2510.18395/) (2025)
- [Rethinking of AlphaStar](https://mlsanity.com/paper/2108.03452/) (2021)
- [A Survey for Deep Reinforcement Learning Based Network Intrusion Detection](https://mlsanity.com/paper/2410.07612/) (2024)
- [Overcoming Environmental Meta-Stationarity in MARL via Adaptive Curriculum and Counterfactual Group Advantage](https://mlsanity.com/paper/2506.07548/) (2025)
- [Applying supervised and reinforcement learning methods to create neural-network-based agents for playing StarCraft II](https://mlsanity.com/paper/2109.12691/) (2021)
- [Causality-Driven Reinforcement Learning for Joint Communication and Sensing](https://mlsanity.com/paper/2409.15329/) (2024)
- [Two-Bridge: Exclusive Objectives and Extended Horizon StarCraft II Benchmark](https://mlsanity.com/paper/2603.06608/) (2026)
- [Compiled Agency: Frontier General-Purpose Coding Agents Build Winning Game Players from Bare Interaction - from Flappy Bird to StarCraft II and Civilization](https://mlsanity.com/paper/2609.18996/) (2026)

## Abstract (arXiv)

Starcraft II (SC2) is widely considered as the most challenging Real Time Strategy (RTS) game. The underlying challenges include a large observation space, a huge (continuous and infinite) action space, partial observations, simultaneous move for all players, and long horizon delayed rewards for local decisions. To push the frontier of AI research, Deepmind and Blizzard jointly developed the StarCraft II Learning Environment (SC2LE) as a testbench of complex decision making systems. SC2LE provides a few mini games such as MoveToBeacon, CollectMineralShards, and DefeatRoaches, where some AI agents have achieved the performance level of human professional players. However, for full games, the current AI agents are still far from achieving human professional level performance. To bridge this gap, we present two full game AI agents in this paper - the AI agent TStarBot1 is based on deep reinforcement learning over a flat action structure, and the AI agent TStarBot2 is based on hard-coded rules over a hierarchical action structure. Both TStarBot1 and TStarBot2 are able to defeat the built-in AI agents from level 1 to level 10 in a full game (1v1 Zerg-vs-Zerg game on the AbyssalReef map), noting that level 8, level 9, and level 10 are cheating agents with unfair advantages such as full vision on the whole map and resource harvest boosting. To the best of our knowledge, this is the first public work to investigate AI agents that can defeat the built-in AI in the StarCraft II full game.