mlsanityData to 7 Oct 2026

Dos and Don'ts of Machine Learning in Computer Security

arXiv 2010.09470 ↗First posted 19 Oct 2020cs.CR cs.LGcited by 98

Not on the map. Why?

Abstract (arXiv)

With the growing processing power of computing systems and the increasing availability of massive datasets, machine learning algorithms have led to major breakthroughs in many different areas. This development has influenced computer security, spawning a series of work on learning-based security systems, such as for malware detection, vulnerability discovery, and binary code analysis. Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance and render learning-based systems potentially unsuitable for security tasks and practical deployment. In this paper, we look at this problem with critical eyes. First, we identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We conduct a study of 30 papers from top-tier security conferences within the past 10 years, confirming that these pitfalls are widespread in the current security literature. In an empirical analysis, we further demonstrate how individual pitfalls can lead to unrealistic performance and interpretations, obstructing the understanding of the security problem at hand. As a remedy, we propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible. Furthermore, we identify open problems when applying machine learning in security and provide directions for further research.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 14; 2024: 16; 2025: 25; 2026: 40.

98 papers.

Sort:

By area:

  1. 2021How to avoid machine learning pitfalls: a guide for academic researcherscited by 34
  2. 2023Uncovering the Limits of Machine Learning for Automatic Vulnerability Detectioncited by 30
  3. 2023Kairos: Practical Intrusion Detection and Investigation using Whole-system Provenancecited by 30
  4. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  5. 2025From CVE Entries to Verifiable Exploits: An Automated Multi-Agent Framework for Reproducing CVEscited by 17
  6. 2023Illuminati: Towards Explaining Graph Neural Networks for Cybersecurity Analysiscited by 14
  7. 2023Decoding the Secrets of Machine Learning in Malware Classification: A Deep Dive into Datasets, Feature Extraction, and Model Performancecited by 12
  8. 2023Efficient Query-Based Attack against ML-Based Android Malware Detection under Zero Knowledge Settingcited by 10
  9. 2024TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time (Extended Version)cited by 9
  10. 2023LogPr'ecis: Unleashing Language Models for Automated Malicious Log Analysiscited by 8
  11. 2024Adversarial Robustness of AI-Generated Image Detectors in the Real Worldcited by 6
  12. 2025When AI Meets the Web: Prompt Injection Risks in Third-Party AI Chatbot Pluginscited by 6
38 more
  1. 2024Assessing the Impact of Packing on Machine Learning-Based Malware Detection and Classification Systemscited by 5
  2. 2025Trace Gadgets: Minimizing Code Context for Machine Learning-Based Vulnerability Predictioncited by 5
  3. 2022Fantomas: Understanding Face Anonymization Reversibilitycited by 4
  4. 2024Enhancing Source Code Security with LLMs: Demystifying The Challenges and Generating Reliable Repairscited by 4
  5. 2024LLM Detectors Still Fall Short of Real World: Case of LLM-Generated Short News-Like Postscited by 4
  6. 2025A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarkscited by 4
  7. 2026Adversarial News and Lost Profits: Manipulating Headlines in LLM-Driven Algorithmic Tradingcited by 4
  8. 2026SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecuritycited by 4
  9. 2024Bi-Directional Transformers vs. word2vec: Discovering Vulnerabilities in Lifted Compiled Codecited by 2
  10. 2025Defending against Adversarial Malware Attacks on ML-based Android Malware Detection Systemscited by 2
  11. 2025Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunitiescited by 2
  12. 2025Demystifying Network Foundation Modelscited by 2
  13. 2023Reassessing feature-based Android malware detection in a contemporary contextcited by 1
  14. 2023Group-based Robustness: A General Framework for Customized Robustness in the Real Worldcited by 1
  15. 2023Testing the Consistency of Performance Scores Reported for Binary Classification Problemscited by 1
  16. 2023Light up that Droid! On the Effectiveness of Static Analysis Features against App Obfuscation for Android Malware Detectioncited by 1
  17. 2023IoTGeM: Generalizable Models for Behaviour-Based IoT Attack Detectioncited by 1
  18. 2024A Training Rate and Survival Heuristic for Inference and Robustness Evaluation (TRASHFIRE)cited by 1
  19. 2024Single-GPU GNN Systems: Traps and Pitfallscited by 1
  20. 2024SISSA: Real-time Monitoring of Hardware Functional Safety and Cybersecurity with In-vehicle SOME/IP Ethernet Trafficcited by 1
  21. 2024R+R: Revisiting Static Feature-Based Android Malware Detection using Machine Learningcited by 1
  22. 2024A Novel Perturb-ability Score to Mitigate Evasion Adversarial Attacks on Flow-Based ML-NIDScited by 1
  23. 2024Comparing Unidirectional, Bidirectional, and Word2vec Models for Discovering Vulnerabilities in Compiled Lifted Codecited by 1
  24. 2025Designing a reliable lateral movement detector using a graph foundation modelcited by 1
  25. 2025Empirical Quantification of Spurious Correlations in Malware Detectioncited by 1
  26. 2025VOLTRON: Detecting Unknown Malware Using Graph-Based Zero-Shot Learningcited by 1
  27. 2025Demystifying the Role of Rule-based Detection in AI Systems for Windows Malware Detectioncited by 1
  28. 2025HiGraph: A Large-Scale Hierarchical Graph Dataset for Malware Analysiscited by 1
  29. 2026Supporting Artifact Evaluation with LLMs: A Study with Published Security Research Paperscited by 1
  30. 2026Towards Reliable Local Security Agents: Verifiable Post-Training for Linux Privilege Escalationcited by 1
  31. 2026CASCADE: A Component Ablation and Corpus Audit of a Layered Local Defense for MCP-Based Systemscited by 1
  32. 2026SoK: Where Do Flow Labels Come From? Auditing Label Provenance in Encrypted Traffic Benchmarkscited by 1
  33. 2026SENTINEL-RL: Offloading Topological Reasoning from LLM Agents in the Security Operations Centercited by 1
  34. 2022SimCLF: A Simple Contrastive Learning Framework for Function-level Binary Embeddingscited by 0
  35. 2023Effectiveness of Moving Target Defenses for Adversarial Attacks in ML-based Malware Detectioncited by 0
  36. 2023Towards machine learning guided by best practicescited by 0
  37. 2023Is there a Trojan! : Literature survey and critical evaluation of the latest ML based modern intrusion detection systems in IoT environmentscited by 0
  38. 2024Improving Adversarial Robustness in Android Malware Detection by Reducing the Impact of Spurious Correlationscited by 0

50 most cited of 98 All 98 (JSON)