# Multi-Agent Penetration Testing AI for the Web

arXiv: https://arxiv.org/abs/2508.20816 · HTML: https://mlsanity.com/paper/2508.20816/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediation](https://mlsanity.com/paper/2407.17788/) (2024)
- [Pythia: Grammar-Based Fuzzing of REST APIs with Coverage-guided Feedback and Learning-based Mutations](https://mlsanity.com/paper/2005.11498/) (2020)

## Built on by (most cited first, 20 of 20)

- [A Survey on Agentic Security: Applications, Threats and Defenses](https://mlsanity.com/paper/2510.06445/) (2025)
- [Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing](https://mlsanity.com/paper/2512.09882/) (2025)
- [What Makes a Good LLM Agent for Real-world Penetration Testing?](https://mlsanity.com/paper/2602.17622/) (2026)
- [Hackers or Hallucinators? A Comprehensive Analysis of LLM-Based Automated Penetration Testing](https://mlsanity.com/paper/2604.05719/) (2026)
- [Towards Optimal Agentic Architectures for Offensive Security Tasks](https://mlsanity.com/paper/2604.18718/) (2026)
- [Calibrated Decision Models for Autonomous Penetration-Testing Harnesses: JEV and Laya as System One Decision Layers for LLM-Driven Pentest Agents](https://mlsanity.com/paper/2609.28940/) (2026)
- [CrackMeBench: Binary Reverse Engineering for Agents](https://mlsanity.com/paper/2605.10597/) (2026)
- [Autonomous LLM Agents & CTFs: A Second Look](https://mlsanity.com/paper/2605.21497/) (2026)
- [Baselines Before Architecture: Evaluating Coding Agents for Autonomous Penetration Testing](https://mlsanity.com/paper/2607.13085/) (2026)
- [Reflection-Driven Control for Trustworthy Code Agents](https://mlsanity.com/paper/2512.21354/) (2025)
- [AWE: Adaptive Agents for Dynamic Web Penetration Testing](https://mlsanity.com/paper/2603.00960/) (2026)
- [Alignment Contracts for Agentic Security Systems](https://mlsanity.com/paper/2605.00081/) (2026)
- [Agentic Fuzzing: Opportunities and Challenges](https://mlsanity.com/paper/2605.10074/) (2026)
- [From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation](https://mlsanity.com/paper/2609.22664/) (2026)
- [Patch2Vuln: Agentic Reconstruction of Vulnerabilities from Linux Distribution Binary Patches](https://mlsanity.com/paper/2605.06601/) (2026)
- [From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World](https://mlsanity.com/paper/2605.10834/) (2026)
- [Benchmarking Mythos-Linked Bug Rediscovery](https://mlsanity.com/paper/2605.17416/) (2026)
- [Transferable Self-Evolving Playbooks for Agentic Security Auditing](https://mlsanity.com/paper/2606.16420/) (2026)
- [Mako: A Self-Evolving Agentic Operating System (SE-AOS) for Autonomous Web Exploitation](https://mlsanity.com/paper/2607.11288/) (2026)
- [SysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary system](https://mlsanity.com/paper/2608.15012/) (2026)

## Abstract (arXiv)

AI-powered development platforms are making software creation accessible to a broader audience, but this democratization has triggered a scalability crisis in security auditing. With studies showing that up to 40% of AI-generated code contains vulnerabilities, the pace of development now vastly outstrips the capacity for thorough security assessment. We present MAPTA, a multi-agent system for autonomous web application security assessment that combines large language model orchestration with tool-grounded execution and end-to-end exploit validation. On the 104-challenge XBOW benchmark, MAPTA achieves 76.9% overall success with perfect performance on SSRF and misconfiguration vulnerabilities, 83% success on broken authorization, and strong results on injection attacks including server-side template injection (85%) and SQL injection (83%). Cross-site scripting (57%) and blind SQL injection (0%) remain challenging. Our comprehensive cost analysis across all challenges totals 21.38 with a median cost of 0.073 for successful attempts versus 0.357 for failures. Success correlates strongly with resource efficiency, enabling practical early-stopping thresholds at approximately 40 tool calls or 0.30 per challenge. MAPTA's real-world findings are impactful given both the popularity of the respective scanned GitHub repositories (8K-70K stars) and MAPTA's low average operating cost of 3.67 per open-source assessment: MAPTA discovered critical vulnerabilities including RCEs, command injections, secret exposure, and arbitrary file write vulnerabilities. Findings are responsibly disclosed, 10 findings are under CVE review.