# AutoPentester: An LLM Agent-based Framework for Automated Pentesting

arXiv: https://arxiv.org/abs/2510.05605 · HTML: https://mlsanity.com/paper/2510.05605/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 7 Oct 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 10 of 10)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks](https://mlsanity.com/paper/2403.01038/) (2024)
- [PentestAgent: Incorporating LLM Agents to Automated Penetration Testing](https://mlsanity.com/paper/2411.05185/) (2024)
- [PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediation](https://mlsanity.com/paper/2407.17788/) (2024)
- [Deep hierarchical reinforcement agents for automated penetration testing](https://mlsanity.com/paper/2109.06449/) (2021)
- [LLMs Killed the Script Kiddie: How Agents Supported by Large Language Models Change the Landscape of Network Threat Testing](https://mlsanity.com/paper/2310.06936/) (2023)

## Built on by (most cited first, 6 of 6)

- [Patch-to-PoC: A Systematic Study of Agentic LLM Systems for Linux Kernel N-Day Reproduction](https://mlsanity.com/paper/2602.07287/) (2026)
- [Automated Penetration Testing with LLM Agents and Classical Planning](https://mlsanity.com/paper/2512.11143/) (2025)
- [uGen: An Agentic Framework for Generating Microarchitectural Attack PoCs](https://mlsanity.com/paper/2605.15503/) (2026)
- [AI-Generated PowerShell Malware: An Experimental Framework and Dataset](https://mlsanity.com/paper/2606.30819/) (2026)
- [VEXAIoT: Autonomous IoT Vulnerability EXploitation using AI Agents](https://mlsanity.com/paper/2607.09653/) (2026)
- [PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs](https://mlsanity.com/paper/2609.18120/) (2026)

## Abstract (arXiv)

Penetration testing and vulnerability assessment are essential industry practices for safeguarding computer systems. As cyber threats grow in scale and complexity, the demand for pentesting has surged, surpassing the capacity of human professionals to meet it effectively. With advances in AI, particularly Large Language Models (LLMs), there have been attempts to automate the pentesting process. However, existing tools such as PentestGPT are still semi-manual, requiring significant professional human interaction to conduct pentests. To this end, we propose a novel LLM agent-based framework, AutoPentester, which automates the pentesting process. Given a target IP, AutoPentester automatically conducts pentesting steps using common security tools in an iterative process. It can dynamically generate attack strategies based on the tool outputs from the previous iteration, mimicking the human pentester approach. We evaluate AutoPentester using Hack The Box and custom-made VMs, comparing the results with the state-of-the-art PentestGPT. Results show that AutoPentester achieves a 27.0% better subtask completion rate and 39.5% more vulnerability coverage with fewer steps. Most importantly, it requires significantly fewer human interactions and interventions compared to PentestGPT. Furthermore, we recruit a group of security industry professional volunteers for a user survey and perform a qualitative analysis to evaluate AutoPentester against industry practices and compare it with PentestGPT. On average, AutoPentester received a score of 3.93 out of 5 based on user reviews, which was 19.8% higher than PentestGPT.