# Benchmarking Software Vulnerability Detection Techniques: A Survey

arXiv: https://arxiv.org/abs/2303.16362 · HTML: https://mlsanity.com/paper/2303.16362/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2023.

## Built on by (most cited first, 4 of 4)

- [Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code](https://mlsanity.com/paper/2311.07989/) (2023)
- [VADER: A Human-Evaluated Benchmark for Vulnerability Assessment, Detection, Explanation, and Remediation](https://mlsanity.com/paper/2505.19395/) (2025)
- [Direction for Detection: A Survey of Automated Vulnerability Detection and all of its Pain Points](https://mlsanity.com/paper/2412.11194/) (2024)
- [VulReaD: Knowledge-Graph-guided Software Vulnerability Reasoning and Detection](https://mlsanity.com/paper/2602.10787/) (2026)

## Abstract (arXiv)

Software vulnerabilities can have serious consequences, which is why many techniques have been proposed to defend against them. Among these, vulnerability detection techniques are a major area of focus. However, there is a lack of a comprehensive approach for benchmarking these proposed techniques. In this paper, we present the first survey that comprehensively investigates and summarizes the current state of software vulnerability detection benchmarking. We review the current literature on benchmarking vulnerability detection, including benchmarking approaches in technique-proposing papers and empirical studies. We also separately discuss the benchmarking approaches for traditional and deep learning-based vulnerability detection techniques. Our survey analyzes the challenges of benchmarking software vulnerability detection techniques and the difficulties involved. We summarize the challenges of benchmarking software vulnerability detection techniques and describe possible solutions for addressing these challenges.