# Towards Making Deep Learning-based Vulnerability Detectors Robust

arXiv: https://arxiv.org/abs/2108.00669 · HTML: https://mlsanity.com/paper/2108.00669/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 73 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 20 of 20)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Adversarial Examples: Attacks and Defenses for Deep Learning](https://mlsanity.com/paper/1712.07107/) (2017)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition](https://mlsanity.com/paper/1903.10346/) (2019)
- [Automated Vulnerability Detection in Source Code Using Deep Representation Learning](https://mlsanity.com/paper/1807.04320/) (2018)
- [SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities](https://mlsanity.com/paper/1807.06756/) (2018)
- [Adversarial Examples for Models of Code](https://mlsanity.com/paper/1910.07517/) (2019)
- [The Art, Science, and Engineering of Fuzzing: A Survey](https://mlsanity.com/paper/1812.00140/) (2018)
- [Adversarial Robustness for Code](https://mlsanity.com/paper/2002.04694/) (2020)
- [Misleading Authorship Attribution of Source Code using Adversarial Learning](https://mlsanity.com/paper/1905.12386/) (2019)
- [Adversarial Deep Ensemble: Evasion Attacks and Defenses for Malware Detection](https://mlsanity.com/paper/2006.16545/) (2020)
- [On Training Robust PDF Malware Classifiers](https://mlsanity.com/paper/1904.03542/) (2019)
- [Learning to Repair Software Vulnerabilities with Generative Adversarial Networks](https://mlsanity.com/paper/1805.07475/) (2018)
- [Defense Against the Dark Arts: An overview of adversarial example security research and future research directions](https://mlsanity.com/paper/1806.04169/) (2018)
- [COSET: A Benchmark for Evaluating Neural Program Embeddings](https://mlsanity.com/paper/1905.11445/) (2019)
- [Robustness to Programmable String Transformations via Augmented Abstract Training](https://mlsanity.com/paper/2002.09579/) (2020)
- [Towards Vulnerability Discovery Using Staged Program Analysis](https://mlsanity.com/paper/1508.04627/) (2015)

## Built on by (most cited first, 4 of 4)

- [Uncovering the Limits of Machine Learning for Automatic Vulnerability Detection](https://mlsanity.com/paper/2306.17193/) (2023)
- [An Unbiased Transformer Source Code Learning with Semantic Vulnerability Graph](https://mlsanity.com/paper/2304.11072/) (2023)
- [Enhancing Source Code Security with LLMs: Demystifying The Challenges and Generating Reliable Repairs](https://mlsanity.com/paper/2409.00571/) (2024)
- [Unintentional Security Flaws in Code: Automated Defense via Root Cause Analysis](https://mlsanity.com/paper/2409.00199/) (2024)

## Abstract (arXiv)

Automatically detecting software vulnerabilities in source code is an important problem that has attracted much attention. In particular, deep learning-based vulnerability detectors, or DL-based detectors, are attractive because they do not need human experts to define features or patterns of vulnerabilities. However, such detectors' robustness is unclear. In this paper, we initiate the study in this aspect by demonstrating that DL-based detectors are not robust against simple code transformations, dubbed attacks in this paper, as these transformations may be leveraged for malicious purposes. As a first step towards making DL-based detectors robust against such attacks, we propose an innovative framework, dubbed ZigZag, which is centered at (i) decoupling feature learning and classifier learning and (ii) using a ZigZag-style strategy to iteratively refine them until they converge to robust features and robust classifiers. Experimental results show that the ZigZag framework can substantially improve the robustness of DL-based detectors.