mlsanityData to 14 Oct 2026

From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEs

arXiv 2509.01835 ↗First posted 1 Sep 2025cs.CRcited by 18

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

High-quality datasets of real-world vulnerabilities and their corresponding verifiable exploits are crucial resources in software security research. Yet such resources remain scarce, as their creation demands intensive manual effort and deep security expertise. In this paper, we present CVE-GENIE, an automated, large language model (LLM)-based multi-agent framework designed to reproduce real-world vulnerabilities, provided in Common Vulnerabilities and Exposures (CVE) format, to enable creation of high-quality vulnerability datasets. Given a CVE entry as input, CVE-GENIE gathers the relevant resources of the CVE, automatically reconstructs the vulnerable environment, and (re)produces a verifiable exploit. Our systematic evaluation highlights the efficiency and robustness of CVE-GENIE's design and successfully reproduces approximately 51% (428 of 841) CVEs published in 2024-2025, complete with their verifiable exploits, at an average cost of 2.77 per CVE. Our pipeline offers a robust method to generate reproducible CVE benchmarks, valuable for diverse applications such as fuzzer evaluation, vulnerability patching, and assessing AI's security capabilities.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 37 of its 59 references that are in the map. First posted on arXiv 1 Sep 2025.

Builds on

37 of 59 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities21 Mar

Other references · 36

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Program Synthesis with Large Language Modelscited by 3,637
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  7. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
  8. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 333
  9. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 272
  10. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 155
  11. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
  12. 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 139
24 more
  1. 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
  2. 2018Automated Vulnerability Detection in Source Code Using Deep Representation Learningcited by 114
  3. 2021Examining Zero-Shot Vulnerability Repair with Large Language Modelscited by 105
  4. 2022Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Modelscited by 103
  5. 2020Dos and Don'ts of Machine Learning in Computer Securitycited by 102
  6. 2023DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detectioncited by 102
  7. 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 97
  8. 2018SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilitiescited by 92
  9. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 86
  10. 2023Impact of Code Language Models on Automated Program Repaircited by 85
  11. 2024Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generationcited by 64
  12. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  13. 2024CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Modelscited by 58
  14. 2024Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Taskscited by 52
  15. 2022VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detectioncited by 51
  16. 2021D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysiscited by 49
  17. 2023ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We?cited by 44
  18. 2024SWE-bench-java: A GitHub Issue Resolving Benchmark for Javacited by 34
  19. 2023Uncovering the Limits of Machine Learning for Automatic Vulnerability Detectioncited by 30
  20. 2024Top Score on the Wrong Exam: On Benchmarking in Machine Learning for Vulnerability Detectioncited by 28
  21. 2025PatchPilot: A Cost-Efficient Software Engineering Agent with Early Attempts on Formal Verificationcited by 22
  22. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  23. 2022VUDENC: Vulnerability Detection with Deep Learning on a Natural Codebase for Pythoncited by 17
  24. 2024Automated Code-centric Software Vulnerability Assessment: How Far Are We? An Empirical Study in C/C++cited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 16.

18 papers.

Sort:

By area:

  1. 2025A Survey on Agentic Security: Applications, Threats and Defensescited by 20
  2. 2025Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testingcited by 15
  3. 2026CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerabilitycited by 9
  4. 2026SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?cited by 9
  5. 2026Patch-to-PoC: A Systematic Study of Agentic LLM Systems for Linux Kernel N-Day Reproductioncited by 7
  6. 2026Multi-Agent Taint Specification Extraction for Vulnerability Detectioncited by 3
  7. 2026PoC-Adapt: Semantic-Aware Automated Vulnerability Reproduction with LLM Multi-Agents and Reinforcement Learning-Driven Adaptive Policycited by 1
  8. 2026Argus: Reorchestrating Static Analysis via a Multi-Agent Ensemble for Full-Chain Security Vulnerability Detectioncited by 1
  9. 2026Small, Free, and Effective: Orchestrating Open-Weight Small Language Models to Outperform Single LLM for Malware Analysiscited by 1
  10. 2026RangeFactory: Scalable Construction of Multi-Hop Cyber Rangescited by 1
  11. 2026Verify Before You Fix: Agentic Execution Grounding for Trustworthy Cross-Language Code Analysiscited by 0
  12. 2026uGen: An Agentic Framework for Generating Microarchitectural Attack PoCscited by 0
6 more
  1. 2026FORGE: Multi-Agent Graduated Exploitation and Detection Engineeringcited by 0
  2. 2026Understanding Implicit Trust Errors in Core Carrier Networks through Multi-Agent Flaw Discovery and Analysiscited by 0
  3. 2026CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Trainingcited by 0
  4. 2026SelfOp: An Optimization Algorithm for Self-Improving Security Agentscited by 0
  5. 2026SyzHarness: Patch-Based Kernel Bug Reproduction with LLM-Synthesized Fuzzing Harnessescited by 0
  6. 2026P2Patch: Exploit-Guided Vulnerability Repaircited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Dual-Loop Agent Framework for Automated Vulnerability Reproduction
  2. 2025AutoPatch: Multi-Agent Framework for Patching Real-World CVE Vulnerabilities
  3. 2025Automated Vulnerability Validation and Verification: A Large Language Model Approach
  4. 2026A Multi-Agent Framework for Automated Exploit Generation with Constraint-Guided Comprehension and Reflection
  5. 2026From Runnable to Verifiable: An Independent Reproducibility Study of LLM/Agent-Driven Vulnerability Validation Artifacts
  6. 2025LLM Agents for Automated Web Vulnerability Reproduction: Are We There Yet?
  7. 2026AXE: Grey-Box Exploitability Confirmation for Localized Vulnerability Reports
  8. 2025A Systematic Study on Generating Web Vulnerability Proof-of-Concepts Using Large Language Models