# Towards security defect prediction with AI

arXiv: https://arxiv.org/abs/1808.09897 · HTML: https://mlsanity.com/paper/1808.09897/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2018.

## Built on by (most cited first, 6 of 6)

- [D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis](https://mlsanity.com/paper/2102.07995/) (2021)
- [Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?](https://mlsanity.com/paper/2306.01754/) (2023)
- [Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation](https://mlsanity.com/paper/2109.03341/) (2021)
- [Study of Distractors in Neural Models of Code](https://mlsanity.com/paper/2303.01739/) (2023)
- [Data-Driven AI Model Signal-Awareness Enhancement and Introspection](https://mlsanity.com/paper/2111.05827/) (2021)
- [ActiveClean: Generating Line-Level Vulnerability Data via Active Learning](https://mlsanity.com/paper/2312.01588/) (2023)

## Abstract (arXiv)

In this study, we investigate the limits of the current state of the art AI system for detecting buffer overflows and compare it with current static analysis tools. To do so, we developed a code generator, s-bAbI, capable of producing an arbitrarily large number of code samples of controlled complexity. We found that the static analysis engines we examined have good precision, but poor recall on this dataset, except for a sound static analyzer that has good precision and recall. We found that the state of the art AI system, a memory network modeled after Choi et al. [1], can achieve similar performance to the static analysis engines, but requires an exhaustive amount of training data in order to do so. Our work points towards future approaches that may solve these problems; namely, using representations of code that can capture appropriate scope information and using deep learning methods that are able to perform arithmetic operations.