# Automated Cyber Defense with Generalizable Graph-based Reinforcement Learning Agents

arXiv: https://arxiv.org/abs/2509.16151 · HTML: https://mlsanity.com/paper/2509.16151/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Learning Combinatorial Optimization Algorithms over Graphs](https://mlsanity.com/paper/1704.01665/) (2017)
- [struc2vec: Learning Node Representations from Structural Identity](https://mlsanity.com/paper/1704.03165/) (2017)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case](https://mlsanity.com/paper/1910.07421/) (2019)
- [Relational inductive bias for physical construction in humans and machines](https://mlsanity.com/paper/1806.01203/) (2018)
- [Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policies](https://mlsanity.com/paper/2211.15557/) (2022)
- [Automated Cyber Defence: A Review](https://mlsanity.com/paper/2303.04926/) (2023)
- [Autonomous Network Defence using Reinforcement Learning](https://mlsanity.com/paper/2409.18197/) (2024)
- [Crown Jewels Analysis using Reinforcement Learning with Attack Graphs](https://mlsanity.com/paper/2108.09358/) (2021)
- [A Comparative Study of Deep Reinforcement Learning Models: DQN vs PPO vs A2C](https://mlsanity.com/paper/2407.14151/) (2024)
- [Incorporating Deception into CyberBattleSim for Autonomous Defense](https://mlsanity.com/paper/2108.13980/) (2021)
- [A Model-Based, Decision-Theoretic Perspective on Automated Cyber Response](https://mlsanity.com/paper/2002.08957/) (2020)
- [Dynamic Network Reconfiguration for Entropy Maximization using Deep Reinforcement Learning](https://mlsanity.com/paper/2205.13578/) (2022)

## Built on by (most cited first, 7 of 7)

- [Building Better Environments for Autonomous Cyber Defence](https://mlsanity.com/paper/2604.08805/) (2026)
- [ACDZero: MCTS Agent for Mastering Automated Cyber Defense](https://mlsanity.com/paper/2601.02196/) (2026)
- [A Red Teaming Framework for Evaluating Robustness of AI-enabled Security Orchestration, Automation, and Response Systems](https://mlsanity.com/paper/2605.17075/) (2026)
- [Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)](https://mlsanity.com/paper/2608.04317/) (2026)
- [Safety-Contract Graph Multi-Agent Reinforcement Learning for Autonomous Network Security Response](https://mlsanity.com/paper/2606.13832/) (2026)
- [CyberWorld: World Models for Sample-Efficient Autonomous Cyber Defense](https://mlsanity.com/paper/2609.31893/) (2026)
- [No One Architecture Fits All: A Cross-Environment Evaluation of Hierarchical Red Team Agents](https://mlsanity.com/paper/2610.00557/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (RL) is emerging as a viable strategy for automated cyber defense (ACD). The traditional RL approach represents networks as a list of computers in various states of safety or threat. Unfortunately, these models are forced to overfit to specific network topologies, rendering them ineffective when faced with even small environmental perturbations. In this work, we frame ACD as a two-player context-based partially observable Markov decision problem with observations represented as attributed graphs. This approach allows our agents to reason through the lens of relational inductive bias. Agents learn how to reason about hosts interacting with other system entities in a more general manner, and their actions are understood as edits to the graph representing the environment. By introducing this bias, we will show that our agents can better reason about the states of networks and zero-shot adapt to new ones. We show that this approach outperforms the state-of-the-art by a wide margin, and makes our agents capable of defending never-before-seen networks against a wide range of adversaries in a variety of complex, and multi-agent environments.