# From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEs

arXiv: https://arxiv.org/abs/2509.01835 · HTML: https://mlsanity.com/paper/2509.01835/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 37 of its 59 references that are in the map.
- First posted on arXiv 1 Sep 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 30 of 37)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents](https://mlsanity.com/paper/2306.03314/) (2023)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead](https://mlsanity.com/paper/2404.04834/) (2024)
- [Vulnerability Detection with Code Language Models: How Far Are We?](https://mlsanity.com/paper/2403.18624/) (2024)
- [Practical Program Repair in the Era of Large Pre-trained Language Models](https://mlsanity.com/paper/2210.14179/) (2022)
- [Automated Vulnerability Detection in Source Code Using Deep Representation Learning](https://mlsanity.com/paper/1807.04320/) (2018)
- [Examining Zero-Shot Vulnerability Repair with Large Language Models](https://mlsanity.com/paper/2112.02125/) (2021)
- [Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models](https://mlsanity.com/paper/2212.14834/) (2022)
- [Dos and Don'ts of Machine Learning in Computer Security](https://mlsanity.com/paper/2010.09470/) (2020)
- [DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection](https://mlsanity.com/paper/2304.00409/) (2023)
- [LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks](https://mlsanity.com/paper/2312.12575/) (2023)
- [CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities](https://mlsanity.com/paper/2503.17332/) (2025)
- [SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities](https://mlsanity.com/paper/1807.06756/) (2018)
- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [Impact of Code Language Models on Automated Program Repair](https://mlsanity.com/paper/2302.05020/) (2023)
- [Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation](https://mlsanity.com/paper/2410.02725/) (2024)
- [CoTexT: Multi-task Learning with Code-Text Transformer](https://mlsanity.com/paper/2105.08645/) (2021)
- [CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Models](https://mlsanity.com/paper/2408.01605/) (2024)
- [Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks](https://mlsanity.com/paper/2407.00121/) (2024)
- [VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detection](https://mlsanity.com/paper/2205.12424/) (2022)
- [D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis](https://mlsanity.com/paper/2102.07995/) (2021)
- [ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We?](https://mlsanity.com/paper/2310.09810/) (2023)

## Built on by (most cited first, 18 of 18)

- [A Survey on Agentic Security: Applications, Threats and Defenses](https://mlsanity.com/paper/2510.06445/) (2025)
- [Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing](https://mlsanity.com/paper/2512.09882/) (2025)
- [CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerability](https://mlsanity.com/paper/2602.03012/) (2026)
- [SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?](https://mlsanity.com/paper/2605.26548/) (2026)
- [Patch-to-PoC: A Systematic Study of Agentic LLM Systems for Linux Kernel N-Day Reproduction](https://mlsanity.com/paper/2602.07287/) (2026)
- [Multi-Agent Taint Specification Extraction for Vulnerability Detection](https://mlsanity.com/paper/2601.10865/) (2026)
- [PoC-Adapt: Semantic-Aware Automated Vulnerability Reproduction with LLM Multi-Agents and Reinforcement Learning-Driven Adaptive Policy](https://mlsanity.com/paper/2604.06618/) (2026)
- [Argus: Reorchestrating Static Analysis via a Multi-Agent Ensemble for Full-Chain Security Vulnerability Detection](https://mlsanity.com/paper/2604.06633/) (2026)
- [Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysis](https://mlsanity.com/paper/2607.20216/) (2026)
- [RangeFactory: Scalable Construction of Multi-Hop Cyber Ranges](https://mlsanity.com/paper/2608.09526/) (2026)
- [Verify Before You Fix: Agentic Execution Grounding for Trustworthy Cross-Language Code Analysis](https://mlsanity.com/paper/2604.10800/) (2026)
- [uGen: An Agentic Framework for Generating Microarchitectural Attack PoCs](https://mlsanity.com/paper/2605.15503/) (2026)
- [FORGE: Multi-Agent Graduated Exploitation and Detection Engineering](https://mlsanity.com/paper/2606.03453/) (2026)
- [Understanding Implicit Trust Errors in Core Carrier Networks through Multi-Agent Flaw Discovery and Analysis](https://mlsanity.com/paper/2607.10315/) (2026)
- [CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Training](https://mlsanity.com/paper/2608.06471/) (2026)
- [SelfOp: An Optimization Algorithm for Self-Improving Security Agents](https://mlsanity.com/paper/2609.22792/) (2026)
- [SyzHarness: Patch-Based Kernel Bug Reproduction with LLM-Synthesized Fuzzing Harnesses](https://mlsanity.com/paper/2609.23889/) (2026)
- [P2Patch: Exploit-Guided Vulnerability Repair](https://mlsanity.com/paper/2610.20245/) (2026)

## Abstract (arXiv)

High-quality datasets of real-world vulnerabilities and their corresponding verifiable exploits are crucial resources in software security research. Yet such resources remain scarce, as their creation demands intensive manual effort and deep security expertise. In this paper, we present CVE-GENIE, an automated, large language model (LLM)-based multi-agent framework designed to reproduce real-world vulnerabilities, provided in Common Vulnerabilities and Exposures (CVE) format, to enable creation of high-quality vulnerability datasets. Given a CVE entry as input, CVE-GENIE gathers the relevant resources of the CVE, automatically reconstructs the vulnerable environment, and (re)produces a verifiable exploit. Our systematic evaluation highlights the efficiency and robustness of CVE-GENIE's design and successfully reproduces approximately 51% (428 of 841) CVEs published in 2024-2025, complete with their verifiable exploits, at an average cost of 2.77 per CVE. Our pipeline offers a robust method to generate reproducible CVE benchmarks, valuable for diverse applications such as fuzzer evaluation, vulnerability patching, and assessing AI's security capabilities.