# Toward Scalable Automated Repository-Level Datasets for Software Vulnerability Detection

arXiv: https://arxiv.org/abs/2603.17974 · HTML: https://mlsanity.com/paper/2603.17974/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 11 references that are in the map.
- First posted on arXiv 18 Mar 2026.

## Builds on (most cited first, 8 of 8)

- [Vulnerability Detection with Code Language Models: How Far Are We?](https://mlsanity.com/paper/2403.18624/) (2024)
- [CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities](https://mlsanity.com/paper/2503.17332/) (2025)
- [BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems](https://mlsanity.com/paper/2505.15216/) (2025)
- [ReposVul: A Repository-Level High-Quality Vulnerability Dataset](https://mlsanity.com/paper/2401.13169/) (2024)
- [Top Score on the Wrong Exam: On Benchmarking in Machine Learning for Vulnerability Detection](https://mlsanity.com/paper/2408.12986/) (2024)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [VulEval: Towards Repository-Level Evaluation of Software Vulnerability Detection](https://mlsanity.com/paper/2404.15596/) (2024)
- [Revisiting the Performance of Deep Learning-Based Vulnerability Detection on Realistic Datasets](https://mlsanity.com/paper/2407.03093/) (2024)

## Built on by (most cited first, 1 of 1)

- [Diagnosing Generalization Boundaries in AI-Based Vulnerability Detection via Calibrated Embedding Distances](https://mlsanity.com/paper/2610.14296/) (2026)

## Abstract (arXiv)

Software vulnerabilities continue to grow in volume and remain difficult to detect in practice. Although learning-based vulnerability detection has progressed, existing benchmarks are largely function-centric and fail to capture realistic, executable, interprocedural settings. Recent repo-level security benchmarks demonstrate the importance of realistic environments, but their manual curation limits scale. This doctoral research proposes an automated benchmark generator that injects realistic vulnerabilities into real-world repositories and synthesizes reproducible proof-of-vulnerability (PoV) exploits, enabling precisely labeled datasets for training and evaluating repo-level vulnerability detection agents. We further investigate an adversarial co-evolution loop between injection and detection agents to improve robustness under realistic constraints.