# NASimEmu: Network Attack Simulator & Emulator for Training Agents Generalizing to Novel Scenarios

arXiv: https://arxiv.org/abs/2305.17246 · HTML: https://mlsanity.com/paper/2305.17246/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)
- [Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulation](https://mlsanity.com/paper/2207.12355/) (2022)
- [Autonomous Penetration Testing using Reinforcement Learning](https://mlsanity.com/paper/1905.05965/) (2019)
- [Finding Effective Security Strategies through Reinforcement Learning and Self-Play](https://mlsanity.com/paper/2009.08120/) (2020)
- [Network Environment Design for Autonomous Cyberdefense](https://mlsanity.com/paper/2103.07583/) (2021)
- [Learning Intrusion Prevention Policies through Optimal Stopping](https://mlsanity.com/paper/2106.07160/) (2021)
- [CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems](https://mlsanity.com/paper/2109.03331/) (2021)
- [Behaviour-Diverse Automatic Penetration Testing: A Curiosity-Driven Multi-Objective Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2202.10630/) (2022)

## Built on by (most cited first, 20 of 20)

- [Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments](https://mlsanity.com/paper/2308.12086/) (2023)
- [Large Language Models are Autonomous Cyber Defenders](https://mlsanity.com/paper/2505.04843/) (2025)
- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [Entity-based Reinforcement Learning for Autonomous Cyber Defence](https://mlsanity.com/paper/2410.17647/) (2024)
- [Scalable Learning of Intrusion Responses through Recursive Decomposition](https://mlsanity.com/paper/2309.03292/) (2023)
- [Intrusion Tolerance for Networked Systems through Two-Level Feedback Control](https://mlsanity.com/paper/2404.01741/) (2024)
- [The Path To Autonomous Cyber Defense](https://mlsanity.com/paper/2404.10788/) (2024)
- [Evaluation of Reinforcement Learning for Autonomous Penetration Testing using A3C, Q-learning and DQN](https://mlsanity.com/paper/2407.15656/) (2024)
- [Building Better Environments for Autonomous Cyber Defence](https://mlsanity.com/paper/2604.08805/) (2026)
- [Learning Robust Penetration Testing Policies under Partial Observability: A systematic evaluation](https://mlsanity.com/paper/2509.20008/) (2025)
- [A Survey for Deep Reinforcement Learning Based Network Intrusion Detection](https://mlsanity.com/paper/2410.07612/) (2024)
- [Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications](https://mlsanity.com/paper/2505.19837/) (2025)
- [Training RL Agents for Multi-Objective Network Defense Tasks](https://mlsanity.com/paper/2505.22531/) (2025)
- [Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agents](https://mlsanity.com/paper/2603.10041/) (2026)
- [NASimJax: A GPU-Accelerated Policy Learning Framework for Penetration Testing](https://mlsanity.com/paper/2603.19864/) (2026)
- [Closing the Sim-to-Real Gap: An Evaluation Framework for Autonomous Cyber Defense Configuration of Commercial EDR](https://mlsanity.com/paper/2606.08168/) (2026)
- [COHORT: Collaborative Orchestration for Hardening via Offensive Replay on Emulated Topologies](https://mlsanity.com/paper/2606.30479/) (2026)
- [NashDreamer: Model-Based Reinforcement Learning for Zero-Sum Imperfect-Information Games](https://mlsanity.com/paper/2609.01549/) (2026)
- [The MAL Simulator: Cyber Operations Simulation based on Attack & Defense Graphs](https://mlsanity.com/paper/2609.16563/) (2026)
- [CyberWorld: World Models for Sample-Efficient Autonomous Cyber Defense](https://mlsanity.com/paper/2609.31893/) (2026)

## Abstract (arXiv)

Current frameworks for training offensive penetration testing agents with deep reinforcement learning struggle to produce agents that perform well in real-world scenarios, due to the reality gap in simulation-based frameworks and the lack of scalability in emulation-based frameworks. Additionally, existing frameworks often use an unrealistic metric that measures the agents' performance on the training data. NASimEmu, a new framework introduced in this paper, addresses these issues by providing both a simulator and an emulator with a shared interface. This approach allows agents to be trained in simulation and deployed in the emulator, thus verifying the realism of the used abstraction. Our framework promotes the development of general agents that can transfer to novel scenarios unseen during their training. For the simulation part, we adopt an existing simulator NASim and enhance its realism. The emulator is implemented with industry-level tools, such as Vagrant, VirtualBox, and Metasploit. Experiments demonstrate that a simulation-trained agent can be deployed in emulation, and we show how to use the framework to train a general agent that transfers into novel, structurally different scenarios. NASimEmu is available as open-source.