# Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design

arXiv: https://arxiv.org/abs/2504.10112 · HTML: https://mlsanity.com/paper/2504.10112/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 24 of 24)

- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models](https://mlsanity.com/paper/2312.04724/) (2023)
- [LLM Agents can Autonomously Exploit One-day Vulnerabilities](https://mlsanity.com/paper/2404.08144/) (2024)
- [AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks](https://mlsanity.com/paper/2403.01038/) (2024)
- [Teams of LLM Agents can Exploit Zero-Day Vulnerabilities](https://mlsanity.com/paper/2406.01637/) (2024)
- [NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security](https://mlsanity.com/paper/2406.05590/) (2024)
- [CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models](https://mlsanity.com/paper/2404.13161/) (2024)
- [Getting pwn'd by AI: Penetration Testing with Large Language Models](https://mlsanity.com/paper/2308.00121/) (2023)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Models](https://mlsanity.com/paper/2408.01605/) (2024)
- [HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testing](https://mlsanity.com/paper/2412.01778/) (2024)
- [An Empirical Evaluation of LLMs for Solving Offensive Security Challenges](https://mlsanity.com/paper/2402.11814/) (2024)
- [A Comprehensive Overview of Large Language Models (LLMs) for Cyber Defences: Opportunities and Directions](https://mlsanity.com/paper/2405.14487/) (2024)
- [VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2501.13411/) (2025)
- [PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediation](https://mlsanity.com/paper/2407.17788/) (2024)
- [LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks](https://mlsanity.com/paper/2310.11409/) (2023)
- [AutoPenBench: Benchmarking Generative Agents for Penetration Testing](https://mlsanity.com/paper/2410.03225/) (2024)
- [Can LLMs Hack Enterprise Networks? Autonomous Assumed Breach Penetration-Testing Active Directory Networks](https://mlsanity.com/paper/2502.04227/) (2025)
- [Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvements](https://mlsanity.com/paper/2410.17141/) (2024)
- [RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents](https://mlsanity.com/paper/2502.16730/) (2025)
- [Review of Generative AI Methods in Cybersecurity](https://mlsanity.com/paper/2403.08701/) (2024)
- [Understanding Hackers' Work: An Empirical Study of Offensive Security Practitioners](https://mlsanity.com/paper/2308.07057/) (2023)
- [AutoPT: How Far Are We from the End2End Automated Web Penetration Testing?](https://mlsanity.com/paper/2411.01236/) (2024)

## Built on by (most cited first, 14 of 14)

- [Autonomous Agents on Blockchains: Standards, Execution Models, and Trust Boundaries](https://mlsanity.com/paper/2601.04583/) (2026)
- [ZeroDayBench: Evaluating LLM Agents on Unseen Zero-Day Vulnerabilities for Cyberdefense](https://mlsanity.com/paper/2603.02297/) (2026)
- [CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking](https://mlsanity.com/paper/2602.08023/) (2026)
- [Cybersecurity AI: Evaluating Agentic Cybersecurity in Attack/Defense CTFs](https://mlsanity.com/paper/2510.17521/) (2025)
- [CritBench: A Framework for Evaluating Cybersecurity Capabilities of Large Language Models in IEC 61850 Digital Substation Environments](https://mlsanity.com/paper/2604.06019/) (2026)
- [How Reliable Are AI Attackers Against a Fixed Vulnerable Target? A 400-Run Empirical Study of LLM Penetration Testing Consistency](https://mlsanity.com/paper/2605.30096/) (2026)
- [Decoupling Reconnaissance and Exploitation: Measuring the Capability Boundaries of LLM-Based Web Penetration Testing](https://mlsanity.com/paper/2606.25332/) (2026)
- [How Do LLM Agents Actually Get the Flag? Trace-Level Provenance for Agentic Offensive Security Evaluation](https://mlsanity.com/paper/2608.26237/) (2026)
- [Can AI Lower the Barrier to Cybersecurity? A Human-Centered Mixed-Methods Study of Novice CTF Learning](https://mlsanity.com/paper/2602.18172/) (2026)
- [From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World](https://mlsanity.com/paper/2605.10834/) (2026)
- [Determinants and Limits of LLM Security-Tool Orchestration: A Study with HexStrike-AI](https://mlsanity.com/paper/2607.02873/) (2026)
- [Evaluating Agentic Learning Harness Capabilities Without Labels via the Scaling Hypothesis](https://mlsanity.com/paper/2608.13608/) (2026)
- [Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectives](https://mlsanity.com/paper/2609.16694/) (2026)
- [Does AI Help Cyber Attackers or Defenders? Evidence from Nonpublic Vulnerabilities and Subsequent Attacks](https://mlsanity.com/paper/2610.06584/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have emerged as a powerful approach for driving offensive penetration-testing tooling. Due to the opaque nature of LLMs, empirical methods are typically used to analyze their efficacy. The quality of this analysis is highly dependent on the chosen testbed, captured metrics and analysis methods employed. This paper analyzes the methodology and benchmarking practices used for evaluating Large Language Model (LLM)-driven attacks, focusing on offensive uses of LLMs in cybersecurity. We review 19 research papers detailing 18 prototypes and their respective testbeds. We detail our findings and provide actionable recommendations for future research, emphasizing the importance of extending existing testbeds, creating baselines, and including comprehensive metrics and qualitative analysis. We also note the distinction between security research and practice, suggesting that CTF-based challenges may not fully represent real-world penetration testing scenarios.