mlsanityData to 14 Oct 2026

Multi-Agent Penetration Testing AI for the Web

arXiv 2508.20816 ↗First posted 28 Aug 2025cs.CR cs.AIcited by 20

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

AI-powered development platforms are making software creation accessible to a broader audience, but this democratization has triggered a scalability crisis in security auditing. With studies showing that up to 40% of AI-generated code contains vulnerabilities, the pace of development now vastly outstrips the capacity for thorough security assessment. We present MAPTA, a multi-agent system for autonomous web application security assessment that combines large language model orchestration with tool-grounded execution and end-to-end exploit validation. On the 104-challenge XBOW benchmark, MAPTA achieves 76.9% overall success with perfect performance on SSRF and misconfiguration vulnerabilities, 83% success on broken authorization, and strong results on injection attacks including server-side template injection (85%) and SQL injection (83%). Cross-site scripting (57%) and blind SQL injection (0%) remain challenging. Our comprehensive cost analysis across all challenges totals 21.38 with a median cost of 0.073 for successful attempts versus 0.357 for failures. Success correlates strongly with resource efficiency, enabling practical early-stopping thresholds at approximately 40 tool calls or 0.30 per challenge. MAPTA's real-world findings are impactful given both the popularity of the respective scanned GitHub repositories (8K-70K stars) and MAPTA's low average operating cost of 3.67 per open-source assessment: MAPTA discovered critical vulnerabilities including RCEs, command injections, secret exposure, and arbitrary file write vulnerabilities. Findings are responsibly disclosed, 10 findings are under CVE review.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 7 of its 21 references that are in the map. First posted on arXiv 28 Aug 2025.

Builds on

7 of 21 references.

References · 7

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  4. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  5. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 68
  6. 2024PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediationcited by 30
  7. 2020Pythia: Grammar-Based Fuzzing of REST APIs with Coverage-guided Feedback and Learning-based Mutationscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 17.

20 papers.

Sort:

By area:

  1. 2025A Survey on Agentic Security: Applications, Threats and Defensescited by 20
  2. 2025Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testingcited by 15
  3. 2026What Makes a Good LLM Agent for Real-world Penetration Testing?cited by 10
  4. 2026Hackers or Hallucinators? A Comprehensive Analysis of LLM-Based Automated Penetration Testingcited by 4
  5. 2026Towards Optimal Agentic Architectures for Offensive Security Taskscited by 4
  6. 2026Calibrated Decision Models for Autonomous Penetration-Testing Harnesses: JEV and Laya as System One Decision Layers for LLM-Driven Pentest Agentscited by 4
  7. 2026CrackMeBench: Binary Reverse Engineering for Agentscited by 2
  8. 2026Autonomous LLM Agents & CTFs: A Second Lookcited by 2
  9. 2026Baselines Before Architecture: Evaluating Coding Agents for Autonomous Penetration Testingcited by 2
  10. 2025Reflection-Driven Control for Trustworthy Code Agentscited by 1
  11. 2026AWE: Adaptive Agents for Dynamic Web Penetration Testingcited by 1
  12. 2026Alignment Contracts for Agentic Security Systemscited by 1
8 more
  1. 2026Agentic Fuzzing: Opportunities and Challengescited by 1
  2. 2026From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementationcited by 1
  3. 2026Patch2Vuln: Agentic Reconstruction of Vulnerabilities from Linux Distribution Binary Patchescited by 0
  4. 2026From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-Worldcited by 0
  5. 2026Benchmarking Mythos-Linked Bug Rediscoverycited by 0
  6. 2026Transferable Self-Evolving Playbooks for Agentic Security Auditingcited by 0
  7. 2026Mako: A Self-Evolving Agentic Operating System (SE-AOS) for Autonomous Web Exploitationcited by 0
  8. 2026SysEvolve: An AI-native, safe, autonomous adversarial attack-defense co-evolutionary systemcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026APT-Agent: Automated Penetration Testing using Large Language Models
  2. 2024AutoPenBench: Benchmarking Generative Agents for Penetration Testing
  3. 2025DoomArena: A framework for Testing AI Agents Against Evolving Security Threats
  4. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks
  5. 2025Penetration Testing of Agentic AI: A Comparative Security Analysis Across Models and Frameworks
  6. 2026Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Reflective Verification and Knowledge-augmented Interaction
  7. 2026PenForge: On-the-Fly Expert Agent Construction for Automated Penetration Testing
  8. 2025xOffense: An Autonomous Multi-Agent Framework for Penetration Testing with Domain-Adapted Large Language Models