# Good News for Script Kiddies? Evaluating Large Language Models for Automated Exploit Generation

arXiv: https://arxiv.org/abs/2505.01065 · HTML: https://mlsanity.com/paper/2505.01065/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 17 references that are in the map.
- First posted on arXiv 2 May 2025.

## Builds on (most cited first, 4 of 4)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [Explaining Neural Scaling Laws](https://mlsanity.com/paper/2102.06701/) (2021)
- [A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models](https://mlsanity.com/paper/2402.13457/) (2024)
- [How Can ChatGPT Support Human Security Testers to Help Mitigate Supply Chain Attacks?](https://mlsanity.com/paper/2310.00710/) (2023)

## Built on by (most cited first, 6 of 6)

- [Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization](https://mlsanity.com/paper/2603.26270/) (2026)
- [AutoEG: Exploiting Known Third-Party Vulnerabilities in Black-Box Web Applications](https://mlsanity.com/paper/2604.00704/) (2026)
- [A Multi-Agent Framework for Automated Exploit Generation with Constraint-Guided Comprehension and Reflection](https://mlsanity.com/paper/2604.05130/) (2026)
- [Data-Centric Benchmarking of Exploit Generation in LLMs: Understanding the Impact of Fine-Tuning](https://mlsanity.com/paper/2606.15123/) (2026)
- [From Runnable to Verifiable: An Independent Reproducibility Study of LLM/Agent-Driven Vulnerability Validation Artifacts](https://mlsanity.com/paper/2608.09567/) (2026)
- [AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation](https://mlsanity.com/paper/2608.25667/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable capabilities in code-related tasks, raising concerns about their potential for automated exploit generation (AEG). This paper presents the first systematic study on LLMs' effectiveness in AEG, evaluating both their cooperativeness and technical proficiency. To mitigate dataset bias, we introduce a benchmark with refactored versions of five software security labs. Additionally, we design an LLM-based attacker to systematically prompt LLMs for exploit generation. Our experiments reveal that GPT-4 and GPT-4o exhibit high cooperativeness, comparable to uncensored models, while Llama3 is the most resistant. However, no model successfully generates exploits for refactored labs, though GPT-4o's minimal errors highlight the potential for LLM-driven AEG advancements.