# Enabling A Network AI Gym for Autonomous Cyber Agents

arXiv: https://arxiv.org/abs/2304.01366 · HTML: https://mlsanity.com/paper/2304.01366/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [AWAC: Accelerating Online Reinforcement Learning with Offline Datasets](https://mlsanity.com/paper/2006.09359/) (2020)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Generalization and Regularization in DQN](https://mlsanity.com/paper/1810.00123/) (2018)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)
- [An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare](https://mlsanity.com/paper/2011.11235/) (2020)
- [Network Environment Design for Autonomous Cyberdefense](https://mlsanity.com/paper/2103.07583/) (2021)
- [CybORG: An Autonomous Cyber Operations Research Gym](https://mlsanity.com/paper/2002.10667/) (2020)
- [CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems](https://mlsanity.com/paper/2109.03331/) (2021)

## Built on by (most cited first, 3 of 3)

- [Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2310.05939/) (2023)
- [Training RL Agents for Multi-Objective Network Defense Tasks](https://mlsanity.com/paper/2505.22531/) (2025)
- [A cybersecurity AI agent selection and decision support framework](https://mlsanity.com/paper/2510.01751/) (2025)

## Abstract (arXiv)

This work aims to enable autonomous agents for network cyber operations (CyOps) by applying reinforcement and deep reinforcement learning (RL/DRL). The required RL training environment is particularly challenging, as it must balance the need for high-fidelity, best achieved through real network emulation, with the need for running large numbers of training episodes, best achieved using simulation. A unified training environment, namely the Cyber Gym for Intelligent Learning (CyGIL) is developed where an emulated CyGIL-E automatically generates a simulated CyGIL-S. From preliminary experimental results, CyGIL-S is capable to train agents in minutes compared with the days required in CyGIL-E. The agents trained in CyGIL-S are transferrable directly to CyGIL-E showing full decision proficiency in the emulated "real" network. Enabling offline RL, the CyGIL solution presents a promising direction towards sim-to-real for leveraging RL agents in real-world cyber networks.