# A Multiagent CyberBattleSim for RL Cyber Operation Agents

arXiv: https://arxiv.org/abs/2304.11052 · HTML: https://mlsanity.com/paper/2304.11052/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 8 of 8)

- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [State Representation Learning for Control: An Overview](https://mlsanity.com/paper/1802.04181/) (2018)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)
- [Deep hierarchical reinforcement agents for automated penetration testing](https://mlsanity.com/paper/2109.06449/) (2021)
- [CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems](https://mlsanity.com/paper/2109.03331/) (2021)
- [Incorporating Deception into CyberBattleSim for Autonomous Defense](https://mlsanity.com/paper/2108.13980/) (2021)
- [Decoys in Cybersecurity: An Exploratory Study to Test the Effectiveness of 2-sided Deception](https://mlsanity.com/paper/2108.11037/) (2021)

## Built on by (most cited first, 8 of 8)

- [Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2310.05939/) (2023)
- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [Intrusion Tolerance for Networked Systems through Two-Level Feedback Control](https://mlsanity.com/paper/2404.01741/) (2024)
- [ACDZero: MCTS Agent for Mastering Automated Cyber Defense](https://mlsanity.com/paper/2601.02196/) (2026)
- [Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications](https://mlsanity.com/paper/2505.19837/) (2025)
- [Adversarial Agents For Attacking Inaudible Voice Activated Devices](https://mlsanity.com/paper/2307.12204/) (2023)
- [The MAL Simulator: Cyber Operations Simulation based on Attack & Defense Graphs](https://mlsanity.com/paper/2609.16563/) (2026)
- [Efficient Nash Equilibrium Computation for Cybersecurity Games](https://mlsanity.com/paper/2609.19399/) (2026)

## Abstract (arXiv)

Hardening cyber physical assets is both crucial and labor-intensive. Recently, Machine Learning (ML) in general and Reinforcement Learning RL) more specifically has shown great promise to automate tasks that otherwise would require significant human insight/intelligence. The development of autonomous RL agents requires a suitable training environment that allows us to quickly evaluate various alternatives, in particular how to arrange training scenarios that pit attackers and defenders against each other. CyberBattleSim is a training environment that supports the training of red agents, i.e., attackers. We added the capability to train blue agents, i.e., defenders. The paper describes our changes and reports on the results we obtained when training blue agents, either in isolation or jointly with red agents. Our results show that training a blue agent does lead to stronger defenses against attacks. In particular, training a blue agent jointly with a red agent increases the blue agent's capability to thwart sophisticated red agents.