# Machine Learning (In) Security: A Stream of Problems

arXiv: https://arxiv.org/abs/2010.16045 · HTML: https://mlsanity.com/paper/2010.16045/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [Adversarial Attacks against Windows PE Malware Detection: A Survey of the State-of-the-Art](https://mlsanity.com/paper/2112.12310/) (2021)
- [A Survey on Semi-Supervised Learning for Delayed Partially Labelled Data Streams](https://mlsanity.com/paper/2106.09170/) (2021)
- [Unraveling the Key of Machine Learning-based Android Malware Detection](https://mlsanity.com/paper/2402.02953/) (2024)
- [IncA-DES: An incremental and adaptive dynamic ensemble selection approach using online K-d tree neighborhood search for data streams with concept drift](https://mlsanity.com/paper/2507.12573/) (2025)
- [Cybersecurity is the True Frontier for Generative AI Success or Failure](https://mlsanity.com/paper/2606.28929/) (2026)
- [On the Effectiveness of Adversarial Samples against Ensemble Learning-based Windows PE Malware Detectors](https://mlsanity.com/paper/2309.13841/) (2023)
- [MLRan: A Behavioural Dataset for Ransomware Analysis and Detection](https://mlsanity.com/paper/2505.18613/) (2025)
- [Online AutoML: Evaluating Poisoning Attacks on Cost-Utility Adversarial Training Defence Strategy for IoT Networks](https://mlsanity.com/paper/2610.05810/) (2026)

## Abstract (arXiv)

Machine Learning (ML) has been widely applied to cybersecurity and is considered state-of-the-art for solving many of the open issues in that field. However, it is very difficult to evaluate how good the produced solutions are, since the challenges faced in security may not appear in other areas. One of these challenges is the concept drift, which increases the existing arms race between attackers and defenders: malicious actors can always create novel threats to overcome the defense solutions, which may not consider them in some approaches. Due to this, it is essential to know how to properly build and evaluate an ML-based security solution. In this paper, we identify, detail, and discuss the main challenges in the correct application of ML techniques to cybersecurity data. We evaluate how concept drift, evolution, delayed labels, and adversarial ML impact the existing solutions. Moreover, we address how issues related to data collection affect the quality of the results presented in the security literature, showing that new strategies are needed to improve current solutions. Finally, we present how existing solutions may fail under certain circumstances, and propose mitigations to them, presenting a novel checklist to help the development of future ML solutions for cybersecurity.