# Optimizing Cyber Defense in Dynamic Active Directories through Reinforcement Learning

arXiv: https://arxiv.org/abs/2406.19596 · HTML: https://mlsanity.com/paper/2406.19596/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 28 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Reward Shaping for Happier Autonomous Cyber Security Agents](https://mlsanity.com/paper/2310.13565/) (2023)
- [Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learning](https://mlsanity.com/paper/2312.04940/) (2023)
- [Analysis of Evolutionary Diversity Optimisation for Permutation Problems](https://mlsanity.com/paper/2102.11469/) (2021)
- [Evolutionary Algorithms for Limiting the Effect of Uncertainty for the Knapsack Problem with Stochastic Profits](https://mlsanity.com/paper/2204.05597/) (2022)
- [Network Defense is Not a Game](https://mlsanity.com/paper/2104.10262/) (2021)
- [A Review on C3I Systems' Security: Vulnerabilities, Attacks, and Countermeasures](https://mlsanity.com/paper/2104.11906/) (2021)
- [Practical Fixed-Parameter Algorithms for Defending Active Directory Style Attack Graphs](https://mlsanity.com/paper/2112.13175/) (2021)
- [Defending Active Directory by Combining Neural Network based Dynamic Program and Evolutionary Diversity Optimisation](https://mlsanity.com/paper/2204.03397/) (2022)
- [Scalable Edge Blocking Algorithms for Defending Active Directory Style Attack Graphs](https://mlsanity.com/paper/2212.04326/) (2022)
- [Evolving Reinforcement Learning Environment to Minimize Learner's Achievable Reward: An Application on Hardening Active Directory Systems](https://mlsanity.com/paper/2304.03998/) (2023)
- [Coevolutionary Pareto Diversity Optimization](https://mlsanity.com/paper/2204.05457/) (2022)
- [Evolutionary Diversity Optimisation in Constructing Satisfying Assignments](https://mlsanity.com/paper/2305.11457/) (2023)
- [Limited Query Graph Connectivity Test](https://mlsanity.com/paper/2302.13036/) (2023)
- [Optimizing Cyber Response Time on Temporal Active Directory Networks Using Decoys](https://mlsanity.com/paper/2403.18162/) (2024)

## Built on by (most cited first, 5 of 5)

- [Unveiling the Black Box: A Multi-Layer Framework for Explaining Reinforcement Learning-Based Cyber Agents](https://mlsanity.com/paper/2505.11708/) (2025)
- [SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity](https://mlsanity.com/paper/2602.08690/) (2026)
- [Explainable Autonomous Cyber Defense using Adversarial Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2604.04442/) (2026)
- [Building Better Environments for Autonomous Cyber Defence](https://mlsanity.com/paper/2604.08805/) (2026)
- [Advancing LLM-Based Security Automation with Customized Group Relative Policy Optimization for Zero-Touch Networks](https://mlsanity.com/paper/2512.09485/) (2025)

## Abstract (arXiv)

This paper addresses a significant gap in Autonomous Cyber Operations (ACO) literature: the absence of effective edge-blocking ACO strategies in dynamic, real-world networks. It specifically targets the cybersecurity vulnerabilities of organizational Active Directory (AD) systems. Unlike the existing literature on edge-blocking defenses which considers AD systems as static entities, our study counters this by recognizing their dynamic nature and developing advanced edge-blocking defenses through a Stackelberg game model between attacker and defender. We devise a Reinforcement Learning (RL)-based attack strategy and an RL-assisted Evolutionary Diversity Optimization-based defense strategy, where the attacker and defender improve each other strategy via parallel gameplay. To address the computational challenges of training attacker-defender strategies on numerous dynamic AD graphs, we propose an RL Training Facilitator that prunes environments and neural networks to eliminate irrelevant elements, enabling efficient and scalable training for large graphs. We extensively train the attacker strategy, as a sophisticated attacker model is essential for a robust defense. Our empirical results successfully demonstrate that our proposed approach enhances defender's proficiency in hardening dynamic AD graphs while ensuring scalability for large-scale AD.