# Security Vulnerability Detection Using Deep Learning Natural Language Processing

arXiv: https://arxiv.org/abs/2105.02388 · HTML: https://mlsanity.com/paper/2105.02388/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 6 May 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Visualizing and Understanding Recurrent Networks](https://mlsanity.com/paper/1506.02078/) (2015)
- [Character-Level Language Modeling with Deeper Self-Attention](https://mlsanity.com/paper/1808.04444/) (2018)
- [Automated Vulnerability Detection in Source Code Using Deep Representation Learning](https://mlsanity.com/paper/1807.04320/) (2018)
- [One Model To Learn Them All](https://mlsanity.com/paper/1706.05137/) (2017)
- [Depthwise Separable Convolutions for Neural Machine Translation](https://mlsanity.com/paper/1706.03059/) (2017)
- [Learning to Recognize Actionable Static Code Warnings (is Intrinsically Easy)](https://mlsanity.com/paper/2006.00444/) (2020)
- [Evaluation of Static Analysis Tools for Finding Vulnerabilities in Java and C/C++ Source Code](https://mlsanity.com/paper/1805.09040/) (2018)

## Built on by (most cited first, 12 of 12)

- [Transformer-Based Language Models for Software Vulnerability Detection](https://mlsanity.com/paper/2204.03214/) (2022)
- [SecureFalcon: Are We There Yet in Automated Software Vulnerability Detection with LLMs?](https://mlsanity.com/paper/2307.06616/) (2023)
- [HW-V2W-Map: Hardware Vulnerability to Weakness Mapping Framework for Root Cause Analysis with GPT-assisted Mitigation Suggestion](https://mlsanity.com/paper/2312.13530/) (2023)
- [A Survey of Source Code Representations for Machine Learning-Based Cybersecurity Tasks](https://mlsanity.com/paper/2403.10646/) (2024)
- [VISION: Robust and Interpretable Code Vulnerability Detection Leveraging Counterfactual Augmentation](https://mlsanity.com/paper/2508.18933/) (2025)
- [Comparing Unidirectional, Bidirectional, and Word2vec Models for Discovering Vulnerabilities in Compiled Lifted Code](https://mlsanity.com/paper/2409.17513/) (2024)
- [SynthFix: Adaptive Neuro-Symbolic Code Vulnerability Repair](https://mlsanity.com/paper/2604.17184/) (2026)
- [FastPacket: Towards Pre-trained Packets Embedding based on FastText for next-generation NIDS](https://mlsanity.com/paper/2209.14727/) (2022)
- [Software Vulnerability Prediction Knowledge Transferring Between Programming Languages](https://mlsanity.com/paper/2303.06177/) (2023)
- [An experimental study in Real-time Facial Emotion Recognition on new 3RL dataset](https://mlsanity.com/paper/2304.03064/) (2023)
- [Vulnerability Detection Using Two-Stage Deep Learning Models](https://mlsanity.com/paper/2305.09673/) (2023)
- [Direction for Detection: A Survey of Automated Vulnerability Detection and all of its Pain Points](https://mlsanity.com/paper/2412.11194/) (2024)

## Abstract (arXiv)

Detecting security vulnerabilities in software before they are exploited has been a challenging problem for decades. Traditional code analysis methods have been proposed, but are often ineffective and inefficient. In this work, we model software vulnerability detection as a natural language processing (NLP) problem with source code treated as texts, and address the automated software venerability detection with recent advanced deep learning NLP models assisted by transfer learning on written English. For training and testing, we have preprocessed the NIST NVD/SARD databases and built a dataset of over 100,000 files in C programming language with 123 types of vulnerabilities. The extensive experiments generate the best performance of over 93% accuracy in detecting security vulnerabilities.