# Learning Optimal Defender Strategies for CAGE-2 using a POMDP Model

arXiv: https://arxiv.org/abs/2509.06539 · HTML: https://mlsanity.com/paper/2509.06539/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 8 Sep 2025.

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)
- [Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulation](https://mlsanity.com/paper/2207.12355/) (2022)
- [Partially Observed Markov Decision Processes. Problem Sets and Internet Supplement](https://mlsanity.com/paper/1604.08127/) (2016)
- [On Autonomous Agents in a Cyber Defence Environment](https://mlsanity.com/paper/2309.07388/) (2023)
- [Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policies](https://mlsanity.com/paper/2211.15557/) (2022)
- [Automated Cyber Defence: A Review](https://mlsanity.com/paper/2303.04926/) (2023)
- [Inroads into Autonomous Network Defence using Explained Reinforcement Learning](https://mlsanity.com/paper/2306.09318/) (2023)
- [Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach](https://mlsanity.com/paper/1809.05258/) (2018)
- [Learning Near-Optimal Intrusion Responses Against Dynamic Attackers](https://mlsanity.com/paper/2301.06085/) (2023)
- [Autonomous Network Defence using Reinforcement Learning](https://mlsanity.com/paper/2409.18197/) (2024)
- [Automated Security Response through Online Learning with Adaptive Conjectures](https://mlsanity.com/paper/2402.12499/) (2024)
- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [Learning Intrusion Prevention Policies through Optimal Stopping](https://mlsanity.com/paper/2106.07160/) (2021)

## Built on by (most cited first, 3 of 3)

- [Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP](https://mlsanity.com/paper/2605.16205/) (2026)
- [Learning Intrusion Response Strategies for OT Systems](https://mlsanity.com/paper/2609.10298/) (2026)
- [CyberWorld: World Models for Sample-Efficient Autonomous Cyber Defense](https://mlsanity.com/paper/2609.31893/) (2026)

## Abstract (arXiv)

CAGE-2 is an accepted benchmark for learning and evaluating defender strategies against cyberattacks. It reflects a scenario where a defender agent protects an IT infrastructure against various attacks. Many defender methods for CAGE-2 have been proposed in the literature. In this paper, we construct a formal model for CAGE-2 using the framework of Partially Observable Markov Decision Process (POMDP). Based on this model, we define an optimal defender strategy for CAGE-2 and introduce a method to efficiently learn this strategy. Our method, called BF-PPO, is based on PPO, and it uses particle filter to mitigate the computational complexity due to the large state space of the CAGE-2 model. We evaluate our method in the CAGE-2 CybORG environment and compare its performance with that of CARDIFF, the highest ranked method on the CAGE-2 leaderboard. We find that our method outperforms CARDIFF regarding the learned defender strategy and the required training time.