# VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements

arXiv: https://arxiv.org/abs/2112.10893 · HTML: https://mlsanity.com/paper/2112.10893/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 60 references that are in the map.
- First posted on arXiv 20 Dec 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [Deep Learning based Vulnerability Detection: Are We There Yet?](https://mlsanity.com/paper/2009.07235/) (2020)
- [A Transformer-based Approach for Source Code Summarization](https://mlsanity.com/paper/2005.00653/) (2020)
- [Automated Vulnerability Detection in Source Code Using Deep Representation Learning](https://mlsanity.com/paper/1807.04320/) (2018)
- [SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities](https://mlsanity.com/paper/1807.06756/) (2018)
- [Vulnerability Detection with Fine-grained Interpretations](https://mlsanity.com/paper/2106.10478/) (2021)
- [Structured Neural Summarization](https://mlsanity.com/paper/1811.01824/) (2018)
- [D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis](https://mlsanity.com/paper/2102.07995/) (2021)
- [VulDeeLocator: A Deep Learning-based Fine-grained Vulnerability Detector](https://mlsanity.com/paper/2001.02350/) (2020)
- [Learning to Represent Edits](https://mlsanity.com/paper/1810.13337/) (2018)
- [Exploring Software Naturalness through Neural Language Models](https://mlsanity.com/paper/2006.12641/) (2020)
- [Learning to Fix Build Errors with Graph2Diff Neural Networks](https://mlsanity.com/paper/1911.01205/) (2019)
- [Learning Semantic Program Embeddings with Graph Interval Neural Network](https://mlsanity.com/paper/2005.09997/) (2020)
- [Patching as Translation: the Data and the Metaphor](https://mlsanity.com/paper/2008.10707/) (2020)
- [Do Code Clones Matter?](https://mlsanity.com/paper/1701.05472/) (2017)
- [LEOPARD: Identifying Vulnerable Code for Vulnerability Assessment through Program Metrics](https://mlsanity.com/paper/1901.11479/) (2019)
- [Learning to map source code to software vulnerability using code-as-a-graph](https://mlsanity.com/paper/2006.08614/) (2020)
- [Digital Investigation of PDF Files: Unveiling Traces of Embedded Malware](https://mlsanity.com/paper/1707.05102/) (2017)

## Built on by (most cited first, 20 of 20)

- [An Empirical Study of Deep Learning Models for Vulnerability Detection](https://mlsanity.com/paper/2212.08109/) (2022)
- [Dataflow Analysis-Inspired Deep Learning for Efficient Vulnerability Detection](https://mlsanity.com/paper/2212.08108/) (2022)
- [Correctness Assessment of Code Generated by Large Language Models Using Internal Representations](https://mlsanity.com/paper/2501.12934/) (2025)
- [Graph Neural Networks for Vulnerability Detection: A Counterfactual Explanation](https://mlsanity.com/paper/2404.15687/) (2024)
- [From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security](https://mlsanity.com/paper/2412.15004/) (2024)
- [Using ChatGPT as a Static Application Security Testing Tool](https://mlsanity.com/paper/2308.14434/) (2023)
- [On the Use of Fine-grained Vulnerable Code Statements for Software Vulnerability Assessment Models](https://mlsanity.com/paper/2203.08417/) (2022)
- [An Unbiased Transformer Source Code Learning with Semantic Vulnerability Graph](https://mlsanity.com/paper/2304.11072/) (2023)
- [A Survey of Source Code Representations for Machine Learning-Based Cybersecurity Tasks](https://mlsanity.com/paper/2403.10646/) (2024)
- [AI for DevSecOps: A Landscape and Future Opportunities](https://mlsanity.com/paper/2404.04839/) (2024)
- [Data and Context Matter: Towards Generalizing AI-based Software Vulnerability Detection](https://mlsanity.com/paper/2508.16625/) (2025)
- [Statement-Level Vulnerability Detection: Learning Vulnerability Patterns Through Information Theory and Contrastive Learning](https://mlsanity.com/paper/2209.10414/) (2022)
- [Semantic-Preserving Transformations as Mutation Operators: A Study on Their Effectiveness in Defect Detection](https://mlsanity.com/paper/2503.23448/) (2025)
- [AutoTrace: From Patches to Triggers via Agentic Interprocedural Exploration](https://mlsanity.com/paper/2607.12058/) (2026)
- [Learning to Quantize Vulnerability Patterns and Match to Locate Statement-Level Vulnerabilities](https://mlsanity.com/paper/2306.06109/) (2023)
- [Unintentional Security Flaws in Code: Automated Defense via Root Cause Analysis](https://mlsanity.com/paper/2409.00199/) (2024)
- [CLNX: Bridging Code and Natural Language for C/C++ Vulnerability-Contributing Commits Identification](https://mlsanity.com/paper/2409.07407/) (2024)
- [Weakly Supervised Vulnerability Localization via Multiple Instance Learning](https://mlsanity.com/paper/2509.11312/) (2025)
- [MulVuln: Enhancing Pre-trained LMs with Shared and Language-Specific Knowledge for Multilingual Vulnerability Detection](https://mlsanity.com/paper/2510.04397/) (2025)
- [DCVD: Dual-Channel Cross-Modal Fusion for Joint Vulnerability Detection and Localization](https://mlsanity.com/paper/2605.11015/) (2026)

## Abstract (arXiv)

Automatically locating vulnerable statements in source code is crucial to assure software security and alleviate developers' debugging efforts. This becomes even more important in today's software ecosystem, where vulnerable code can flow easily and unwittingly within and across software repositories like GitHub. Across such millions of lines of code, traditional static and dynamic approaches struggle to scale. Although existing machine-learning-based approaches look promising in such a setting, most work detects vulnerable code at a higher granularity -- at the method or file level. Thus, developers still need to inspect a significant amount of code to locate the vulnerable statement(s) that need to be fixed. This paper presents VELVET, a novel ensemble learning approach to locate vulnerable statements. Our model combines graph-based and sequence-based neural networks to successfully capture the local and global context of a program graph and effectively understand code semantics and vulnerable patterns. To study VELVET's effectiveness, we use an off-the-shelf synthetic dataset and a recently published real-world dataset. In the static analysis setting, where vulnerable functions are not detected in advance, VELVET achieves 4.5x better performance than the baseline static analyzers on the real-world data. For the isolated vulnerability localization task, where we assume the vulnerability of a function is known while the specific vulnerable statement is unknown, we compare VELVET with several neural networks that also attend to local and global context of code. VELVET achieves 99.6% and 43.6% top-1 accuracy over synthetic data and real-world data, respectively, outperforming the baseline deep-learning models by 5.3-29.0%.