# Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulation

arXiv: https://arxiv.org/abs/2207.12355 · HTML: https://mlsanity.com/paper/2207.12355/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 25 Jul 2022.

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)
- [Causal Bayesian Optimization](https://mlsanity.com/paper/2005.11741/) (2020)
- [Network Environment Design for Autonomous Cyberdefense](https://mlsanity.com/paper/2103.07583/) (2021)
- [Probabilistic Evaluation of Sequential Plans from Causal Models with Hidden Variables](https://mlsanity.com/paper/1302.4977/) (2013)
- [Intrusion Prevention through Optimal Stopping](https://mlsanity.com/paper/2111.00289/) (2021)
- [Dynamic Causal Bayesian Optimization](https://mlsanity.com/paper/2110.13891/) (2021)
- [Prospective Artificial Intelligence Approaches for Active Cyber Defence](https://mlsanity.com/paper/2104.09981/) (2021)
- [Deep Reinforcement Learning for Cybersecurity Threat Detection and Protection: A Review](https://mlsanity.com/paper/2206.02733/) (2022)

## Built on by (most cited first, 30 of 37)

- [Automated Cyber Defence: A Review](https://mlsanity.com/paper/2303.04926/) (2023)
- [NASimEmu: Network Attack Simulator & Emulator for Training Agents Generalizing to Novel Scenarios](https://mlsanity.com/paper/2305.17246/) (2023)
- [Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments](https://mlsanity.com/paper/2308.12086/) (2023)
- [Inroads into Autonomous Network Defence using Explained Reinforcement Learning](https://mlsanity.com/paper/2306.09318/) (2023)
- [Learning Near-Optimal Intrusion Responses Against Dynamic Attackers](https://mlsanity.com/paper/2301.06085/) (2023)
- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [Entity-based Reinforcement Learning for Autonomous Cyber Defence](https://mlsanity.com/paper/2410.17647/) (2024)
- [Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learning](https://mlsanity.com/paper/2312.04940/) (2023)
- [Scalable Learning of Intrusion Responses through Recursive Decomposition](https://mlsanity.com/paper/2309.03292/) (2023)
- [Intrusion Tolerance for Networked Systems through Two-Level Feedback Control](https://mlsanity.com/paper/2404.01741/) (2024)
- [Deep Reinforcement Learning for Autonomous Cyber Defence: A Survey](https://mlsanity.com/paper/2310.07745/) (2023)
- [CSLE: A Reinforcement Learning Platform for Autonomous Security Management](https://mlsanity.com/paper/2604.15590/) (2026)
- [Unveiling the Black Box: A Multi-Layer Framework for Explaining Reinforcement Learning-Based Cyber Agents](https://mlsanity.com/paper/2505.11708/) (2025)
- [SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity](https://mlsanity.com/paper/2602.08690/) (2026)
- [Training Automated Defense Strategies Using Graph-based Cyber Attack Simulations](https://mlsanity.com/paper/2304.11084/) (2023)
- [Learning Optimal Defender Strategies for CAGE-2 using a POMDP Model](https://mlsanity.com/paper/2509.06539/) (2025)
- [Beyond Rewards in Reinforcement Learning for Cyber Defence](https://mlsanity.com/paper/2602.04809/) (2026)
- [Building Better Environments for Autonomous Cyber Defence](https://mlsanity.com/paper/2604.08805/) (2026)
- [Multi-Agent Reinforcement Learning for Maritime Operational Technology Cyber Security](https://mlsanity.com/paper/2401.10149/) (2024)
- [Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities](https://mlsanity.com/paper/2509.06921/) (2025)
- [Using Quantum Solved Deep Boltzmann Machines to Increase the Data Efficiency of RL Agents](https://mlsanity.com/paper/2408.17240/) (2024)
- [The Price of Pessimism for Automated Defense](https://mlsanity.com/paper/2409.19237/) (2024)
- [A Survey for Deep Reinforcement Learning Based Network Intrusion Detection](https://mlsanity.com/paper/2410.07612/) (2024)
- [Machine Theory of Mind for Autonomous Cyber-Defence](https://mlsanity.com/paper/2412.04367/) (2024)
- [Less is more? Rewards in RL for Cyber Defence](https://mlsanity.com/paper/2503.03245/) (2025)
- [Training RL Agents for Multi-Objective Network Defense Tasks](https://mlsanity.com/paper/2505.22531/) (2025)
- [Enhancing Cloud Network Resilience via a Robust LLM-Empowered Multi-Agent Reinforcement Learning Framework](https://mlsanity.com/paper/2601.07122/) (2026)
- [Multi-Agent Influence Diagrams to Hybrid Threat Modeling](https://mlsanity.com/paper/2603.03526/) (2026)
- [Towards Type Agnostic Cyber Defense Agents](https://mlsanity.com/paper/2412.01542/) (2024)
- [Graphical Models for Decision-Making: Integrating Causality and Game Theory](https://mlsanity.com/paper/2504.13210/) (2025)

## Abstract (arXiv)

In this paper we explore cyber security defence, through the unification of a novel cyber security simulator with models for (causal) decision-making through optimisation. Particular attention is paid to a recently published approach: dynamic causal Bayesian optimisation (DCBO). We propose that DCBO can act as a blue agent when provided with a view of a simulated network and a causal model of how a red agent spreads within that network. To investigate how DCBO can perform optimal interventions on host nodes, in order to reduce the cost of intrusions caused by the red agent. Through this we demonstrate a complete cyber-simulation system, which we use to generate observational data for DCBO and provide numerical quantitative results which lay the foundations for future work in this space.