# Dos and Don'ts of Machine Learning in Computer Security

arXiv: https://arxiv.org/abs/2010.09470 · HTML: https://mlsanity.com/paper/2010.09470/

## Key facts

- Cited by 102 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 30 of 102)

- [How to avoid machine learning pitfalls: a guide for academic researchers](https://mlsanity.com/paper/2108.02497/) (2021)
- [Kairos: Practical Intrusion Detection and Investigation using Whole-system Provenance](https://mlsanity.com/paper/2308.05034/) (2023)
- [Uncovering the Limits of Machine Learning for Automatic Vulnerability Detection](https://mlsanity.com/paper/2306.17193/) (2023)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEs](https://mlsanity.com/paper/2509.01835/) (2025)
- [Illuminati: Towards Explaining Graph Neural Networks for Cybersecurity Analysis](https://mlsanity.com/paper/2303.14836/) (2023)
- [Decoding the Secrets of Machine Learning in Malware Classification: A Deep Dive into Datasets, Feature Extraction, and Model Performance](https://mlsanity.com/paper/2307.14657/) (2023)
- [Efficient Query-Based Attack against ML-Based Android Malware Detection under Zero Knowledge Setting](https://mlsanity.com/paper/2309.01866/) (2023)
- [TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time (Extended Version)](https://mlsanity.com/paper/2402.01359/) (2024)
- [When AI Meets the Web: Prompt Injection Risks in Third-Party AI Chatbot Plugins](https://mlsanity.com/paper/2511.05797/) (2025)
- [LogPr'ecis: Unleashing Language Models for Automated Malicious Log Analysis](https://mlsanity.com/paper/2307.08309/) (2023)
- [Adversarial Robustness of AI-Generated Image Detectors in the Real World](https://mlsanity.com/paper/2410.01574/) (2024)
- [Assessing the Impact of Packing on Machine Learning-Based Malware Detection and Classification Systems](https://mlsanity.com/paper/2410.24017/) (2024)
- [Trace Gadgets: Minimizing Code Context for Machine Learning-Based Vulnerability Prediction](https://mlsanity.com/paper/2504.13676/) (2025)
- [Fantomas: Understanding Face Anonymization Reversibility](https://mlsanity.com/paper/2210.10651/) (2022)
- [Enhancing Source Code Security with LLMs: Demystifying The Challenges and Generating Reliable Repairs](https://mlsanity.com/paper/2409.00571/) (2024)
- [LLM Detectors Still Fall Short of Real World: Case of LLM-Generated Short News-Like Posts](https://mlsanity.com/paper/2409.03291/) (2024)
- [A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarks](https://mlsanity.com/paper/2506.10502/) (2025)
- [Adversarial News and Lost Profits: Manipulating Headlines in LLM-Driven Algorithmic Trading](https://mlsanity.com/paper/2601.13082/) (2026)
- [SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity](https://mlsanity.com/paper/2602.08690/) (2026)
- [Bi-Directional Transformers vs. word2vec: Discovering Vulnerabilities in Lifted Compiled Code](https://mlsanity.com/paper/2405.20611/) (2024)
- [Defending against Adversarial Malware Attacks on ML-based Android Malware Detection Systems](https://mlsanity.com/paper/2501.13782/) (2025)
- [HiGraph: A Large-Scale Hierarchical Graph Dataset for Malware Analysis](https://mlsanity.com/paper/2509.02113/) (2025)
- [Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities](https://mlsanity.com/paper/2509.06921/) (2025)
- [Demystifying Network Foundation Models](https://mlsanity.com/paper/2509.23089/) (2025)
- [Reassessing feature-based Android malware detection in a contemporary context](https://mlsanity.com/paper/2301.12778/) (2023)
- [Group-based Robustness: A General Framework for Customized Robustness in the Real World](https://mlsanity.com/paper/2306.16614/) (2023)
- [Testing the Consistency of Performance Scores Reported for Binary Classification Problems](https://mlsanity.com/paper/2310.12527/) (2023)
- [Light up that Droid! On the Effectiveness of Static Analysis Features against App Obfuscation for Android Malware Detection](https://mlsanity.com/paper/2310.15645/) (2023)
- [IoTGeM: Generalizable Models for Behaviour-Based IoT Attack Detection](https://mlsanity.com/paper/2401.01343/) (2023)

## Abstract (arXiv)

With the growing processing power of computing systems and the increasing availability of massive datasets, machine learning algorithms have led to major breakthroughs in many different areas. This development has influenced computer security, spawning a series of work on learning-based security systems, such as for malware detection, vulnerability discovery, and binary code analysis. Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance and render learning-based systems potentially unsuitable for security tasks and practical deployment. In this paper, we look at this problem with critical eyes. First, we identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We conduct a study of 30 papers from top-tier security conferences within the past 10 years, confirming that these pitfalls are widespread in the current security literature. In an empirical analysis, we further demonstrate how individual pitfalls can lead to unrealistic performance and interpretations, obstructing the understanding of the security problem at hand. As a remedy, we propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible. Furthermore, we identify open problems when applying machine learning in security and provide directions for further research.