# Network Defense is Not a Game

arXiv: https://arxiv.org/abs/2104.10262 · HTML: https://mlsanity.com/paper/2104.10262/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design](https://mlsanity.com/paper/2012.02096/) (2020)
- [Distributed Prioritized Experience Replay](https://mlsanity.com/paper/1803.00933/) (2018)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [Tactics of Adversarial Attack on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1703.06748/) (2017)
- [Delving into adversarial attacks on deep policies](https://mlsanity.com/paper/1705.06452/) (2017)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Learning to Perform Physics Experiments via Deep Reinforcement Learning](https://mlsanity.com/paper/1611.01843/) (2016)
- [Network Environment Design for Autonomous Cyberdefense](https://mlsanity.com/paper/2103.07583/) (2021)
- [Characterizing Attacks on Deep Reinforcement Learning](https://mlsanity.com/paper/1907.09470/) (2019)
- [Reinforcement Learning for Autonomous Defence in Software-Defined Networking](https://mlsanity.com/paper/1808.05770/) (2018)
- [CybORG: An Autonomous Cyber Operations Research Gym](https://mlsanity.com/paper/2002.10667/) (2020)
- [The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning](https://mlsanity.com/paper/1810.10369/) (2018)
- [SoK: A Survey of Open-Source Threat Emulators](https://mlsanity.com/paper/2003.01518/) (2020)
- [Adversarial Reinforcement Learning under Partial Observability in Autonomous Computer Network Defence](https://mlsanity.com/paper/1902.09062/) (2019)

## Built on by (most cited first, 6 of 6)

- [Deep Reinforcement Learning for Autonomous Cyber Defence: A Survey](https://mlsanity.com/paper/2310.07745/) (2023)
- [Incorporating Deception into CyberBattleSim for Autonomous Defense](https://mlsanity.com/paper/2108.13980/) (2021)
- [Optimizing Cyber Defense in Dynamic Active Directories through Reinforcement Learning](https://mlsanity.com/paper/2406.19596/) (2024)
- [Unveiling the Black Box: A Multi-Layer Framework for Explaining Reinforcement Learning-Based Cyber Agents](https://mlsanity.com/paper/2505.11708/) (2025)
- [Training RL Agents for Multi-Objective Network Defense Tasks](https://mlsanity.com/paper/2505.22531/) (2025)
- [A Framework for Adversarial Analysis of Decision Support Systems Prior to Deployment](https://mlsanity.com/paper/2505.21414/) (2025)

## Abstract (arXiv)

Research seeks to apply Artificial Intelligence (AI) to scale and extend the capabilities of human operators to defend networks. A fundamental problem that hinders the generalization of successful AI approaches -- i.e., beating humans at playing games -- is that network defense cannot be defined as a single game with a fixed set of rules. Our position is that network defense is better characterized as a collection of games with uncertain and possibly drifting rules. Hence, we propose to define network defense tasks as distributions of network environments, to: (i) enable research to apply modern AI techniques, such as unsupervised curriculum learning and reinforcement learning for network defense; and, (ii) facilitate the design of well-defined challenges that can be used to compare approaches for autonomous cyberdefense. To demonstrate that an approach for autonomous network defense is practical it is important to be able to reason about the boundaries of its applicability. Hence, we need to be able to define network defense tasks that capture sets of adversarial tactics, techniques, and procedures (TTPs); quality of service (QoS) requirements; and TTPs available to defenders. Furthermore, the abstractions to define these tasks must be extensible; must be backed by well-defined semantics that allow us to reason about distributions of environments; and should enable the generation of data and experiences from which an agent can learn. Our approach named Network Environment Design for Autonomous Cyberdefense inspired the architecture of FARLAND, a Framework for Advanced Reinforcement Learning for Autonomous Network Defense, which we use at MITRE to develop RL network defenders that perform blue actions from the MITRE Shield matrix against attackers with TTPs that drift from MITRE ATT&CK TTPs.