# Exposing Surveillance Detection Routes via Reinforcement Learning, Attack Graphs, and Cyber Terrain

arXiv: https://arxiv.org/abs/2211.03027 · HTML: https://mlsanity.com/paper/2211.03027/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 6 Nov 2022.

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Autonomous Penetration Testing using Reinforcement Learning](https://mlsanity.com/paper/1905.05965/) (2019)
- [Deep hierarchical reinforcement agents for automated penetration testing](https://mlsanity.com/paper/2109.06449/) (2021)
- [Using Cyber Terrain in Reinforcement Learning for Penetration Testing](https://mlsanity.com/paper/2108.07124/) (2021)
- [Crown Jewels Analysis using Reinforcement Learning with Attack Graphs](https://mlsanity.com/paper/2108.09358/) (2021)
- [Discovering Exfiltration Paths Using Reinforcement Learning with Attack Graphs](https://mlsanity.com/paper/2201.12416/) (2022)
- [A Layered Reference Model for Penetration Testing with Reinforcement Learning and Attack Graphs](https://mlsanity.com/paper/2206.06934/) (2022)
- [Modeling Penetration Testing with Reinforcement Learning Using Capture-the-Flag Challenges: Trade-offs between Model-free Learning and A Priori Knowledge](https://mlsanity.com/paper/2005.12632/) (2020)

## Built on by (most cited first, 3 of 3)

- [Leveraging Reinforcement Learning in Red Teaming for Advanced Ransomware Attack Simulations](https://mlsanity.com/paper/2406.17576/) (2024)
- [Discovering Command and Control (C2) Channels on Tor and Public Networks Using Reinforcement Learning](https://mlsanity.com/paper/2402.09200/) (2024)
- [From Capabilities to Performance: Evaluating Key Functional Properties of LLM Architectures in Penetration Testing](https://mlsanity.com/paper/2509.14289/) (2025)

## Abstract (arXiv)

Reinforcement learning (RL) operating on attack graphs leveraging cyber terrain principles are used to develop reward and state associated with determination of surveillance detection routes (SDR). This work extends previous efforts on developing RL methods for path analysis within enterprise networks. This work focuses on building SDR where the routes focus on exploring the network services while trying to evade risk. RL is utilized to support the development of these routes by building a reward mechanism that would help in realization of these paths. The RL algorithm is modified to have a novel warm-up phase which decides in the initial exploration which areas of the network are safe to explore based on the rewards and penalty scale factor.