# A Practical Approach to the Automatic Classification of Security-Relevant Commits

arXiv: https://arxiv.org/abs/1807.02458 · HTML: https://mlsanity.com/paper/1807.02458/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2018.

## Built on by (most cited first, 13 of 13)

- [CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Software](https://mlsanity.com/paper/2107.08760/) (2021)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [Can Large Language Models Find And Fix Vulnerable Software?](https://mlsanity.com/paper/2308.10345/) (2023)
- [DeepCVA: Automated Commit-level Vulnerability Assessment with Deep Multi-task Learning](https://mlsanity.com/paper/2108.08041/) (2021)
- [VulCurator: A Vulnerability-Fixing Commit Detector](https://mlsanity.com/paper/2209.03260/) (2022)
- [Automatic Data Labeling for Software Vulnerability Prediction Models: How Far Are We?](https://mlsanity.com/paper/2407.17803/) (2024)
- [Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation](https://mlsanity.com/paper/2109.03341/) (2021)
- [Multi-Granularity Detector for Vulnerability Fixes](https://mlsanity.com/paper/2305.13884/) (2023)
- [GitRanking: A Ranking of GitHub Topics for Software Classification using Active Sampling](https://mlsanity.com/paper/2205.09379/) (2022)
- [Automated Mapping of Vulnerability Advisories onto their Fix Commits in Open Source Repositories](https://mlsanity.com/paper/2103.13375/) (2021)
- [Favia: Forensic Agent for Vulnerability-fix Identification and Analysis](https://mlsanity.com/paper/2602.12500/) (2026)
- [Detecting Security Fixes in Open-Source Repositories using Static Code Analyzers](https://mlsanity.com/paper/2105.03346/) (2021)
- [A Comprehensive Evaluation of Code Language Models for Security Patch Detection](https://mlsanity.com/paper/2605.13138/) (2026)

## Abstract (arXiv)

The lack of reliable sources of detailed information on the vulnerabilities of open-source software (OSS) components is a major obstacle to maintaining a secure software supply chain and an effective vulnerability management process. Standard sources of advisories and vulnerability data, such as the National Vulnerability Database (NVD), are known to suffer from poor coverage and inconsistent quality. To reduce our dependency on these sources, we propose an approach that uses machine-learning to analyze source code repositories and to automatically identify commits that are security-relevant (i.e., that are likely to fix a vulnerability). We treat the source code changes introduced by commits as documents written in natural language, classifying them using standard document classification methods. Combining independent classifiers that use information from different facets of commits, our method can yield high precision (80%) while ensuring acceptable recall (43%). In particular, the use of information extracted from the source code changes yields a substantial improvement over the best known approach in state of the art, while requiring a significantly smaller amount of training data and employing a simpler architecture.