# CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerability

arXiv: https://arxiv.org/abs/2602.03012 · HTML: https://mlsanity.com/paper/2602.03012/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 16 of 16)

- [Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents](https://mlsanity.com/paper/2410.02644/) (2024)
- [R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents](https://mlsanity.com/paper/2504.07164/) (2025)
- [Vulnerability Detection with Code Language Models: How Far Are We?](https://mlsanity.com/paper/2403.18624/) (2024)
- [CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities](https://mlsanity.com/paper/2503.17332/) (2025)
- [CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Software](https://mlsanity.com/paper/2107.08760/) (2021)
- [SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks](https://mlsanity.com/paper/2506.11791/) (2025)
- [How Effective Are Neural Networks for Fixing Security Vulnerabilities](https://mlsanity.com/paper/2305.18607/) (2023)
- [Automated Benchmark Generation for Repository-Level Coding Tasks](https://mlsanity.com/paper/2503.07701/) (2025)
- [A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents](https://mlsanity.com/paper/2506.23844/) (2025)
- [PATCHEVAL: A New Benchmark for Evaluating LLMs on Patching Real-World Vulnerabilities](https://mlsanity.com/paper/2511.11019/) (2025)
- [To Err is Machine: Vulnerability Detection Challenges LLM Reasoning](https://mlsanity.com/paper/2403.17218/) (2024)
- [From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEs](https://mlsanity.com/paper/2509.01835/) (2025)
- [SoK: Automated Vulnerability Repair: Methods, Tools, and Assessments](https://mlsanity.com/paper/2506.11697/) (2025)
- [VADER: A Human-Evaluated Benchmark for Vulnerability Assessment, Detection, Explanation, and Remediation](https://mlsanity.com/paper/2505.19395/) (2025)
- [Improving LLM Reasoning for Vulnerability Detection via Group Relative Policy Optimization](https://mlsanity.com/paper/2507.03051/) (2025)
- [Semantics-Aligned, Curriculum-Driven, and Reasoning-Enhanced Vulnerability Repair Framework](https://mlsanity.com/paper/2510.01002/) (2025)

## Built on by (most cited first, 9 of 9)

- [The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence](https://mlsanity.com/paper/2605.26494/) (2026)
- [RangeFactory: Scalable Construction of Multi-Hop Cyber Ranges](https://mlsanity.com/paper/2608.09526/) (2026)
- [SecProbe: Adaptive Evaluation of Coding Agents on Cybersecurity Vulnerabilities](https://mlsanity.com/paper/2609.33763/) (2026)
- [FORGE: Multi-Agent Graduated Exploitation and Detection Engineering](https://mlsanity.com/paper/2606.03453/) (2026)
- [CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild](https://mlsanity.com/paper/2608.23181/) (2026)
- [VulValidate: Auditing Function-Level Vulnerability Labels with Executable Evidence](https://mlsanity.com/paper/2610.05103/) (2026)
- [Last Secure Code Benchmark: Is the Code You Vibe Secure?](https://mlsanity.com/paper/2610.14766/) (2026)
- [CWE-bench: Refocusing Agentic Security from Exploitation to Defense](https://mlsanity.com/paper/2610.17020/) (2026)
- [EvoProbe: Evaluation-Driven Benchmark Evolution for LLM Agents](https://mlsanity.com/paper/2610.19433/) (2026)

## Abstract (arXiv)

Evaluating and improving the security capabilities of code agents requires high-quality, executable vulnerability tasks. However, existing works rely on costly, unscalable manual reproduction and suffer from outdated data distributions. To address these, we present CVE-Factory, the first multi-agent framework to achieve expert-level quality in automatically transforming sparse CVE metadata into fully executable agentic tasks. Cross-validation against human expert reproductions shows that CVE-Factory achieves 95% solution correctness and 96% environment fidelity, confirming its expert-level quality. It is also evaluated on the latest realistic vulnerabilities and achieves a 66.2% verified success. This automation enables two downstream contributions. First, we construct LiveCVEBench, a continuously updated benchmark of 190 tasks spanning 14 languages and 153 repositories that captures emerging threats including AI-tooling vulnerabilities. Second, we synthesize over 1,000 executable training environments, the first large-scale scaling of agentic tasks in code security. Fine-tuned Qwen3-32B improves from 5.3% to 35.8% on LiveCVEBench, surpassing Claude 4.5 Sonnet, with gains generalizing to Terminal Bench (12.5% to 31.3%). We open-source CVE-Factory, LiveCVEBench, Abacus-cve (fine-tuned model), training dataset, and leaderboard. All resources are available at https://github.com/livecvebench/CVE-Factory .