# Dataset Bias in Android Malware Detection

arXiv: https://arxiv.org/abs/2205.15532 · HTML: https://mlsanity.com/paper/2205.15532/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2022.

## Built on by (most cited first, 3 of 3)

- [Security and Privacy Product Inclusion](https://mlsanity.com/paper/2404.13220/) (2024)
- [MH-1M: A 1.34 Million-Sample Comprehensive Multi-Feature Android Malware Dataset for Machine Learning, Deep Learning, Large Language Models, and Threat Intelligence Research](https://mlsanity.com/paper/2511.00342/) (2025)
- [Android Malware Detection: A Machine Leaning Approach](https://mlsanity.com/paper/2511.00894/) (2025)

## Abstract (arXiv)

Researchers have proposed kinds of malware detection methods to solve the explosive mobile security threats. We argue that the experiment results are inflated due to the research bias introduced by the variability of malware dataset. We explore the impact of bias in Android malware detection in three aspects, the method used to flag the ground truth, the distribution of malware families in the dataset, and the methods to use the dataset. We implement a set of experiments of different VT thresholds and find that the methods used to flag the malware data affect the malware detection performance directly. We further compare the impact of malware family types and composition on malware detection in detail. The superiority of each approach is different under various combinations of malware families. Through our extensive experiments, we showed that the methods to use the dataset can have a misleading impact on evaluation, and the performance difference can be up to over 40%. We argue that these research biases observed in this paper should be carefully controlled/eliminated to enforce a fair comparison of malware detection techniques. Providing reasonable and explainable results is better than only reporting a high detection accuracy with vague dataset and experimental settings.