# Penetration Testing == POMDP Solving?

arXiv: https://arxiv.org/abs/1306.4714 · HTML: https://mlsanity.com/paper/1306.4714/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2013.

## Built on by (most cited first, 9 of 9)

- [PentestAgent: Incorporating LLM Agents to Automated Penetration Testing](https://mlsanity.com/paper/2411.05185/) (2024)
- [Simulating SQL Injection Vulnerability Exploitation Using Q-Learning Reinforcement Learning Agents](https://mlsanity.com/paper/2101.03118/) (2021)
- [BertRLFuzzer: A BERT and Reinforcement Learning Based Fuzzer](https://mlsanity.com/paper/2305.12534/) (2023)
- [Automated Penetration Testing with LLM Agents and Classical Planning](https://mlsanity.com/paper/2512.11143/) (2025)
- [APT-Agent: Automated Penetration Testing using Large Language Models](https://mlsanity.com/paper/2605.24949/) (2026)
- [A Unified Modeling Framework for Automated Penetration Testing](https://mlsanity.com/paper/2502.11588/) (2025)
- [STRIATUM-CTF: A Protocol-Driven Agentic Framework for General-Purpose CTF Solving](https://mlsanity.com/paper/2603.22577/) (2026)
- [GAIL-PT: A Generic Intelligent Penetration Testing Framework with Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2204.01975/) (2022)
- [Tutorial on Course-of-Action (COA) Attack Search Methods in Computer Networks](https://mlsanity.com/paper/2205.13763/) (2022)

## Abstract (arXiv)

Penetration Testing is a methodology for assessing network security, by generating and executing possible attacks. Doing so automatically allows for regular and systematic testing without a prohibitive amount of human labor. A key question then is how to generate the attacks. This is naturally formulated as a planning problem. Previous work (Lucangeli et al. 2010) used classical planning and hence ignores all the incomplete knowledge that characterizes hacking. More recent work (Sarraute et al. 2011) makes strong independence assumptions for the sake of scaling, and lacks a clear formal concept of what the attack planning problem actually is. Herein, we model that problem in terms of partially observable Markov decision processes (POMDP). This grounds penetration testing in a well-researched formalism, highlighting important aspects of this problem's nature. POMDPs allow to model information gathering as an integral part of the problem, thus providing for the first time a means to intelligently mix scanning actions with actual exploits.