# Hackers or Hallucinators? A Comprehensive Analysis of LLM-Based Automated Penetration Testing

arXiv: https://arxiv.org/abs/2604.05719 · HTML: https://mlsanity.com/paper/2604.05719/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 136 references that are in the map.
- First posted on arXiv 7 Apr 2026.

## Builds on (most cited first, 30 of 41)

- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models](https://mlsanity.com/paper/2408.08926/) (2024)
- [SoK: Agentic Skills -- Beyond Tool Use in LLM Agents](https://mlsanity.com/paper/2602.20867/) (2026)
- [LLM Agents can Autonomously Exploit One-day Vulnerabilities](https://mlsanity.com/paper/2404.08144/) (2024)
- [From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs](https://mlsanity.com/paper/2504.15965/) (2025)
- [Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward](https://mlsanity.com/paper/2602.12430/) (2026)
- [GUI Agents with Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2411.04890/) (2024)
- [LLM Agents can Autonomously Hack Websites](https://mlsanity.com/paper/2402.06664/) (2024)
- [AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks](https://mlsanity.com/paper/2403.01038/) (2024)
- [CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities](https://mlsanity.com/paper/2503.17332/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testing](https://mlsanity.com/paper/2412.01778/) (2024)
- [An Empirical Evaluation of LLMs for Solving Offensive Security Challenges](https://mlsanity.com/paper/2402.11814/) (2024)
- [VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2501.13411/) (2025)
- [LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks](https://mlsanity.com/paper/2310.11409/) (2023)
- [Graph-based Agent Memory: Taxonomy, Techniques, and Applications](https://mlsanity.com/paper/2602.05665/) (2026)
- [Multi-Agent Penetration Testing AI for the Web](https://mlsanity.com/paper/2508.20816/) (2025)
- [Toward Efficient Agents: Memory, Tool learning, and Planning](https://mlsanity.com/paper/2601.14192/) (2026)
- [CAI: An Open, Bug Bounty-Ready Cybersecurity AI](https://mlsanity.com/paper/2504.06017/) (2025)
- [RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents](https://mlsanity.com/paper/2502.16730/) (2025)
- [On the Surprising Efficacy of LLMs for Penetration-Testing](https://mlsanity.com/paper/2507.00829/) (2025)
- [What Makes a Good LLM Agent for Real-world Penetration Testing?](https://mlsanity.com/paper/2602.17622/) (2026)
- [AutoPentest: Enhancing Vulnerability Management With Autonomous LLM Agents](https://mlsanity.com/paper/2505.10321/) (2025)
- [AutoPT: How Far Are We from the End2End Automated Web Penetration Testing?](https://mlsanity.com/paper/2411.01236/) (2024)

## Built on by (most cited first, 4 of 4)

- [SysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary system](https://mlsanity.com/paper/2608.15012/) (2026)
- [SWE-Test: Benchmarking LLM Vulnerability Discovery via Input Prediction](https://mlsanity.com/paper/2609.06229/) (2026)
- [Big Enough to Break Out: Tracking the Rising Capability of LLM Penetration-Testing Agents](https://mlsanity.com/paper/2609.10780/) (2026)
- [Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectives](https://mlsanity.com/paper/2609.16694/) (2026)

## Abstract (arXiv)

The rapid advancement of Large Language Models (LLMs) has created new opportunities for Automated Penetration Testing (AutoPT), spawning numerous frameworks aimed at achieving end-to-end autonomous attacks. However, despite the proliferation of related studies, existing research generally lacks systematic architectural analysis and large-scale empirical comparisons under a unified benchmark. Therefore, this paper presents the first Systematization of Knowledge (SoK) focusing on the architectural design and comprehensive empirical evaluation of current LLM-based AutoPT frameworks. At systematization level, we comprehensively review existing framework designs across six dimensions: agent architecture, agent plan, agent memory, agent execution, external knowledge, and benchmarks. At empirical level, we conduct large-scale experiments on 13 representative open-source AutoPT frameworks and 2 baseline frameworks utilizing a unified benchmark. The experiments consumed over 10 billion tokens in total and generated more than 1,500 execution logs, which were manually reviewed and analyzed over four months by a panel of more than 15 researchers with expertise in cybersecurity. By investigating the latest progress in this rapidly developing field, we provide researchers with a structured taxonomy to understand existing LLM-based AutoPT frameworks and a large-scale empirical benchmark, along with promising directions for future research.