# Impact of Data Snooping on Deep Learning Models for Locating Vulnerabilities in Lifted Code

arXiv: https://arxiv.org/abs/2412.02048 · HTML: https://mlsanity.com/paper/2412.02048/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 8 of 8)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Dos and Don'ts of Machine Learning in Computer Security](https://mlsanity.com/paper/2010.09470/) (2020)
- [Benchmarking datasets for Anomaly-based Network Intrusion Detection: KDD CUP 99 alternatives](https://mlsanity.com/paper/1811.05372/) (2018)
- [Bin2vec: Learning Representations of Binary Executable Programs for Security Tasks](https://mlsanity.com/paper/2002.03388/) (2020)
- [Deep-Learning-based Vulnerability Detection in Binary Executables](https://mlsanity.com/paper/2212.01254/) (2022)
- [Multimodal Deep Learning for Flaw Detection in Software Programs](https://mlsanity.com/paper/2009.04549/) (2020)
- [Bi-Directional Transformers vs. word2vec: Discovering Vulnerabilities in Lifted Compiled Code](https://mlsanity.com/paper/2405.20611/) (2024)
- [Comparing Unidirectional, Bidirectional, and Word2vec Models for Discovering Vulnerabilities in Compiled Lifted Code](https://mlsanity.com/paper/2409.17513/) (2024)

## Abstract (arXiv)

This study examines the impact of data snooping on neural networks used to detect vulnerabilities in lifted code, and builds on previous research that used word2vec and unidirectional and bidirectional transformer-based embeddings. The research specifically focuses on how model performance is affected when embedding models are trained with datasets, which include samples used for neural network training and validation. The results show that introducing data snooping did not significantly alter model performance, suggesting that data snooping had a minimal impact or that samples randomly dropped as part of the methodology contained hidden features critical to achieving optimal performance. In addition, the findings reinforce the conclusions of previous research, which found that models trained with GPT-2 embeddings consistently outperformed neural networks trained with other embeddings. The fact that this holds even when data snooping is introduced into the embedding model indicates GPT-2's robustness in representing complex code features, even under less-than-ideal conditions.