# Autonomous Network Defence using Reinforcement Learning

arXiv: https://arxiv.org/abs/2409.18197 · HTML: https://mlsanity.com/paper/2409.18197/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 7 of its 15 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 7 of 7)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [Distributed Prioritized Experience Replay](https://mlsanity.com/paper/1803.00933/) (2018)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)

## Built on by (most cited first, 15 of 15)

- [CybORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2410.16324/) (2024)
- [Entity-based Reinforcement Learning for Autonomous Cyber Defence](https://mlsanity.com/paper/2410.17647/) (2024)
- [DRMD: Deep Reinforcement Learning for Malware Detection under Concept Drift](https://mlsanity.com/paper/2508.18839/) (2025)
- [Automated Cyber Defense with Generalizable Graph-based Reinforcement Learning Agents](https://mlsanity.com/paper/2509.16151/) (2025)
- [Designing Robust Cyber-Defense Agents with Evolving Behavior Trees](https://mlsanity.com/paper/2410.16383/) (2024)
- [SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity](https://mlsanity.com/paper/2602.08690/) (2026)
- [Out-of-Distribution Detection for Neurosymbolic Autonomous Cyber Agents](https://mlsanity.com/paper/2412.02875/) (2024)
- [Learning Optimal Defender Strategies for CAGE-2 using a POMDP Model](https://mlsanity.com/paper/2509.06539/) (2025)
- [Building Better Environments for Autonomous Cyber Defence](https://mlsanity.com/paper/2604.08805/) (2026)
- [The Price of Pessimism for Automated Defense](https://mlsanity.com/paper/2409.19237/) (2024)
- [Enhancing Cloud Network Resilience via a Robust LLM-Empowered Multi-Agent Reinforcement Learning Framework](https://mlsanity.com/paper/2601.07122/) (2026)
- [Inherently Interpretable and Uncertainty-Aware Models for Online Learning in Cyber-Security Problems](https://mlsanity.com/paper/2411.09393/) (2024)
- [Strategic Cyber Defense via Reinforcement Learning-Guided Combinatorial Auctions](https://mlsanity.com/paper/2509.10983/) (2025)
- [Learning Red Agent Policy from Observations for Neurosymbolic Autonomous Cyber Agents](https://mlsanity.com/paper/2606.18223/) (2026)
- [REPLICANT: Learning Policies for Evading and Hardening Malware Detectors](https://mlsanity.com/paper/2608.28499/) (2026)

## Abstract (arXiv)

In the network security arms race, the defender is significantly disadvantaged as they need to successfully detect and counter every malicious attack. In contrast, the attacker needs to succeed only once. To level the playing field, we investigate the effectiveness of autonomous agents in a realistic network defence scenario. We first outline the problem, provide the background on reinforcement learning and detail our proposed agent design. Using a network environment simulation, with 13 hosts spanning 3 subnets, we train a novel reinforcement learning agent and show that it can reliably defend continual attacks by two advanced persistent threat (APT) red agents: one with complete knowledge of the network layout and another which must discover resources through exploration but is more general.