# Structural Generalization in Autonomous Cyber Incident Response with Message-Passing Neural Networks and Reinforcement Learning

arXiv: https://arxiv.org/abs/2407.05775 · HTML: https://mlsanity.com/paper/2407.05775/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 12 of 12)

- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [Graph Matching Networks for Learning the Similarity of Graph Structured Objects](https://mlsanity.com/paper/1904.12787/) (2019)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Characteristic Functions on Graphs: Birds of a Feather, from Statistical Descriptors to Parametric Models](https://mlsanity.com/paper/2005.07959/) (2020)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)
- [Finding Effective Security Strategies through Reinforcement Learning and Self-Play](https://mlsanity.com/paper/2009.08120/) (2020)
- [Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policies](https://mlsanity.com/paper/2211.15557/) (2022)
- [Reward Shaping for Happier Autonomous Cyber Security Agents](https://mlsanity.com/paper/2310.13565/) (2023)
- [What Planning Problems Can A Relational Neural Network Solve?](https://mlsanity.com/paper/2312.03682/) (2023)
- [Symbolic Relational Deep Reinforcement Learning based on Graph Neural Networks and Autoregressive Policy Decomposition](https://mlsanity.com/paper/2009.12462/) (2020)

## Built on by (most cited first, 7 of 7)

- [Entity-based Reinforcement Learning for Autonomous Cyber Defence](https://mlsanity.com/paper/2410.17647/) (2024)
- [Building Better Environments for Autonomous Cyber Defence](https://mlsanity.com/paper/2604.08805/) (2026)
- [Vejde: A Framework for Inductive Deep Reinforcement Learning Based on Factor Graph Color Refinement](https://mlsanity.com/paper/2509.09219/) (2025)
- [General Autonomous Cybersecurity Defense: Learning Robust Policies for Dynamic Topologies and Diverse Attackers](https://mlsanity.com/paper/2506.22706/) (2025)
- [STAIR: Effective Incident Response Using an End-to-End Agentic Planning Framework](https://mlsanity.com/paper/2608.09524/) (2026)
- [Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agents](https://mlsanity.com/paper/2603.10041/) (2026)
- [A Cyber Range Evaluation of Autonomous Network Incident Response Agents](https://mlsanity.com/paper/2609.16541/) (2026)

## Abstract (arXiv)

We believe that agents for automated incident response based on machine learning need to handle changes in network structure. Computer networks are dynamic, and can naturally change in structure over time. Retraining agents for small network changes costs time and energy. We attempt to address this issue with an existing method of relational agent learning, where the relations between objects are assumed to remain consistent across problem instances. The state of the computer network is represented as a relational graph and encoded through a message passing neural network. The message passing neural network and an agent policy using the encoding are optimized end-to-end using reinforcement learning. We evaluate the approach on the second instance of the Cyber Autonomy Gym for Experimentation (CAGE~2), a cyber incident simulator that simulates attacks on an enterprise network. We create variants of the original network with different numbers of hosts and agents are tested without additional training on them. Our results show that agents using relational information are able to find solutions despite changes to the network, and can perform optimally in some instances. Agents using the default vector state representation perform better, but need to be specially trained on each network variant, demonstrating a trade-off between specialization and generalization.