mlsanityData to 14 Oct 2026

Using Cyber Terrain in Reinforcement Learning for Penetration Testing

arXiv 2108.07124 ↗First posted 16 Aug 2021cs.LG cs.CR cs.NIcited by 16

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) has been applied to attack graphs for penetration testing, however, trained agents do not reflect reality because the attack graphs lack operational nuances typically captured within the intelligence preparation of the battlefield (IPB) that include notions of (cyber) terrain. In particular, current practice constructs attack graphs exclusively using the Common Vulnerability Scoring System (CVSS) and its components. We present methods for constructing attack graphs using notions from IPB on cyber terrain analysis of obstacles, avenues of approach, key terrain, observation and fields of fire, and cover and concealment. We demonstrate our methods on an example where firewalls are treated as obstacles and represented in (1) the reward space and (2) the state dynamics. We show that terrain analysis can be used to bring realism to attack graphs for RL.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 8 of its 39 references that are in the map. First posted on arXiv 16 Aug 2021.

Builds on

8 of 39 references.

References · 8

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  6. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 157
  7. 2019Autonomous Penetration Testing using Reinforcement Learningcited by 33
  8. 2019Attack Graph Obfuscationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 2; 2024: 4; 2025: 2; 2026: 2.

16 papers.

Sort:

By area:

  1. 2021Intrusion Prevention through Optimal Stoppingcited by 18
  2. 2023Learning Near-Optimal Intrusion Responses Against Dynamic Attackerscited by 15
  3. 2021Crown Jewels Analysis using Reinforcement Learning with Attack Graphscited by 8
  4. 2022Discovering Exfiltration Paths Using Reinforcement Learning with Attack Graphscited by 8
  5. 2022Learning Security Strategies through Game Play and Optimal Stoppingcited by 5
  6. 2024SoK: On the Offensive Potential of AIcited by 5
  7. 2026SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecuritycited by 4
  8. 2022Exposing Surveillance Detection Routes via Reinforcement Learning, Attack Graphs, and Cyber Terraincited by 3
  9. 2023Training Automated Defense Strategies Using Graph-based Cyber Attack Simulationscited by 3
  10. 2024Discovering Command and Control Channels Using Reinforcement Learningcited by 3
  11. 2026Building Better Environments for Autonomous Cyber Defencecited by 3
  12. 2024Leveraging Reinforcement Learning in Red Teaming for Advanced Ransomware Attack Simulationscited by 1
4 more
  1. 2025Training RL Agents for Multi-Objective Network Defense Taskscited by 1
  2. 2025Reinforcement Learning for Automated Cybersecurity Penetration Testingcited by 1
  3. 2022Tutorial on Course-of-Action (COA) Attack Search Methods in Computer Networkscited by 0
  4. 2024Discovering Command and Control (C2) Channels on Tor and Public Networks Using Reinforcement Learningcited by 0