# CSLE: A Reinforcement Learning Platform for Autonomous Security Management

arXiv: https://arxiv.org/abs/2604.15590 · HTML: https://mlsanity.com/paper/2604.15590/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 16 Apr 2026.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 18 of 18)

- [Deep Reinforcement Learning from Self-Play in Imperfect-Information Games](https://mlsanity.com/paper/1603.01121/) (2016)
- [Phasic Policy Gradient](https://mlsanity.com/paper/2009.04416/) (2020)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning](https://mlsanity.com/paper/1801.08917/) (2018)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)
- [ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Tasks](https://mlsanity.com/paper/2502.05352/) (2025)
- [AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds](https://mlsanity.com/paper/2501.06706/) (2025)
- [Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulation](https://mlsanity.com/paper/2207.12355/) (2022)
- [Finding Effective Security Strategies through Reinforcement Learning and Self-Play](https://mlsanity.com/paper/2009.08120/) (2020)
- [Network Environment Design for Autonomous Cyberdefense](https://mlsanity.com/paper/2103.07583/) (2021)
- [Intrusion Prevention through Optimal Stopping](https://mlsanity.com/paper/2111.00289/) (2021)
- [Learning Near-Optimal Intrusion Responses Against Dynamic Attackers](https://mlsanity.com/paper/2301.06085/) (2023)
- [Conjectural Online Learning with First-order Beliefs in Asymmetric Information Stochastic Games](https://mlsanity.com/paper/2402.18781/) (2024)
- [CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems](https://mlsanity.com/paper/2109.03331/) (2021)
- [Scalable Learning of Intrusion Responses through Recursive Decomposition](https://mlsanity.com/paper/2309.03292/) (2023)
- [Intrusion Tolerance for Networked Systems through Two-Level Feedback Control](https://mlsanity.com/paper/2404.01741/) (2024)
- [Adaptive Network Security Policies via Belief Aggregation and Rollout](https://mlsanity.com/paper/2507.15163/) (2025)
- [CyGym: A Simulation-Based Game-Theoretic Analysis Framework for Cybersecurity](https://mlsanity.com/paper/2506.21688/) (2025)

## Built on by (most cited first, 5 of 5)

- [Agentic Incident Response through Digital Twin-Enhanced Multiscale Planning](https://mlsanity.com/paper/2608.02422/) (2026)
- [Hierarchical Agentic Incident Response with Digital-Twin-Validated Attack Inference](https://mlsanity.com/paper/2608.15016/) (2026)
- [A Cyber Range Evaluation of Autonomous Network Incident Response Agents](https://mlsanity.com/paper/2609.16541/) (2026)
- [The MAL Simulator: Cyber Operations Simulation based on Attack & Defense Graphs](https://mlsanity.com/paper/2609.16563/) (2026)
- [Efficient Nash Equilibrium Computation for Cybersecurity Games](https://mlsanity.com/paper/2609.19399/) (2026)

## Abstract (arXiv)

Reinforcement learning is a promising approach to autonomous and adaptive security management in networked systems. However, current reinforcement learning solutions for security management are mostly limited to simulation environments and it is unclear how they generalize to operational systems. In this paper, we address this limitation by presenting CSLE: a reinforcement learning platform for autonomous security management that enables experimentation under realistic conditions. Conceptually, CSLE encompasses two systems. First, it includes an emulation system that replicates key components of the target system in a virtualized environment. We use this system to gather measurements and logs, based on which we identify a system model, such as a Markov decision process. Second, it includes a simulation system where security strategies are efficiently learned through simulations of the system model. The learned strategies are then evaluated and refined in the emulation system to close the gap between theoretical and operational performance. We demonstrate CSLE through four use cases: flow control, replication control, segmentation control, and recovery control. Through these use cases, we show that CSLE enables near-optimal security management in an environment that approximates an operational system.