# Learning Intrusion Prevention Policies through Optimal Stopping

arXiv: https://arxiv.org/abs/2106.07160 · HTML: https://mlsanity.com/paper/2106.07160/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 50 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Partially Observed Markov Decision Processes. Problem Sets and Internet Supplement](https://mlsanity.com/paper/1604.08127/) (2016)
- [Finding Effective Security Strategies through Reinforcement Learning and Self-Play](https://mlsanity.com/paper/2009.08120/) (2020)
- [Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach](https://mlsanity.com/paper/1809.05258/) (2018)
- [Online Reinforcement Learning of Optimal Threshold Policies for Markov Decision Processes](https://mlsanity.com/paper/1912.10325/) (2019)
- [A Supervisory Control Approach to Dynamic Cyber-Security](https://mlsanity.com/paper/1409.0838/) (2014)

## Built on by (most cited first, 11 of 11)

- [Automated Cyber Defence: A Review](https://mlsanity.com/paper/2303.04926/) (2023)
- [NASimEmu: Network Attack Simulator & Emulator for Training Agents Generalizing to Novel Scenarios](https://mlsanity.com/paper/2305.17246/) (2023)
- [Intrusion Prevention through Optimal Stopping](https://mlsanity.com/paper/2111.00289/) (2021)
- [Inroads into Autonomous Network Defence using Explained Reinforcement Learning](https://mlsanity.com/paper/2306.09318/) (2023)
- [Learning Near-Optimal Intrusion Responses Against Dynamic Attackers](https://mlsanity.com/paper/2301.06085/) (2023)
- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [Scalable Learning of Intrusion Responses through Recursive Decomposition](https://mlsanity.com/paper/2309.03292/) (2023)
- [Learning Security Strategies through Game Play and Optimal Stopping](https://mlsanity.com/paper/2205.14694/) (2022)
- [A System for Interactive Examination of Learned Security Policies](https://mlsanity.com/paper/2204.01126/) (2022)
- [Training Automated Defense Strategies Using Graph-based Cyber Attack Simulations](https://mlsanity.com/paper/2304.11084/) (2023)
- [Learning Optimal Defender Strategies for CAGE-2 using a POMDP Model](https://mlsanity.com/paper/2509.06539/) (2025)

## Abstract (arXiv)

We study automated intrusion prevention using reinforcement learning. In a novel approach, we formulate the problem of intrusion prevention as an optimal stopping problem. This formulation allows us insight into the structure of the optimal policies, which turn out to be threshold based. Since the computation of the optimal defender policy using dynamic programming is not feasible for practical cases, we approximate the optimal policy through reinforcement learning in a simulation environment. To define the dynamics of the simulation, we emulate the target infrastructure and collect measurements. Our evaluations show that the learned policies are close to optimal and that they indeed can be expressed using thresholds.