# Reviewer Integration and Performance Measurement for Malware Detection

arXiv: https://arxiv.org/abs/1510.07338 · HTML: https://mlsanity.com/paper/1510.07338/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2015.

## Built on by (most cited first, 19 of 19)

- [Continuous Learning for Android Malware Detection](https://mlsanity.com/paper/2302.04332/) (2023)
- [The Role of Machine Learning in Cybersecurity](https://mlsanity.com/paper/2206.09707/) (2022)
- [LAMD: Context-driven Android Malware Detection and Classification with LLMs](https://mlsanity.com/paper/2502.13055/) (2025)
- [SoK: The Impact of Unlabelled Data in Cyberthreat Detection](https://mlsanity.com/paper/2205.08944/) (2022)
- [Learning Security Classifiers with Verified Global Robustness Properties](https://mlsanity.com/paper/2105.11363/) (2021)
- [SoK: Pragmatic Assessment of Machine Learning for Network Intrusion Detection](https://mlsanity.com/paper/2305.00550/) (2023)
- [Decoding the Secrets of Machine Learning in Malware Classification: A Deep Dive into Datasets, Feature Extraction, and Model Performance](https://mlsanity.com/paper/2307.14657/) (2023)
- [TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time (Extended Version)](https://mlsanity.com/paper/2402.01359/) (2024)
- [The Cross-evaluation of Machine Learning-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2203.04686/) (2022)
- [MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection](https://mlsanity.com/paper/2302.10739/) (2023)
- [DRMD: Deep Reinforcement Learning for Malware Detection under Concept Drift](https://mlsanity.com/paper/2508.18839/) (2025)
- [Towards a Fair Comparison and Realistic Evaluation Framework of Android Malware Detectors based on Static Analysis and Machine Learning](https://mlsanity.com/paper/2205.12569/) (2022)
- [Realizable Universal Adversarial Perturbations for Malware](https://mlsanity.com/paper/2102.06747/) (2021)
- [Unraveling the Key of Machine Learning-based Android Malware Detection](https://mlsanity.com/paper/2402.02953/) (2024)
- [MalMixer: Few-Shot Malware Classification with Retrieval-Augmented Semi-Supervised Learning](https://mlsanity.com/paper/2409.13213/) (2024)
- [On the Effectiveness of Adversarial Training on Malware Classifiers](https://mlsanity.com/paper/2412.18218/) (2024)
- [ADAPT: A Pseudo-labeling Approach to Combat Concept Drift in Malware Detection](https://mlsanity.com/paper/2507.08597/) (2025)
- [Effectiveness of Moving Target Defenses for Adversarial Attacks in ML-based Malware Detection](https://mlsanity.com/paper/2302.00537/) (2023)
- [SEED: Semi-supervised Continual MalwarE Detection for Tackling ConcEpt Drift on a BuDget](https://mlsanity.com/paper/2605.24903/) (2026)

## Abstract (arXiv)

We present and evaluate a large-scale malware detection system integrating machine learning with expert reviewers, treating reviewers as a limited labeling resource. We demonstrate that even in small numbers, reviewers can vastly improve the system's ability to keep pace with evolving threats. We conduct our evaluation on a sample of VirusTotal submissions spanning 2.5 years and containing 1.1 million binaries with 778GB of raw feature data. Without reviewer assistance, we achieve 72% detection at a 0.5% false positive rate, performing comparable to the best vendors on VirusTotal. Given a budget of 80 accurate reviews daily, we improve detection to 89% and are able to detect 42% of malicious binaries undetected upon initial submission to VirusTotal. Additionally, we identify a previously unnoticed temporal inconsistency in the labeling of training datasets. We compare the impact of training labels obtained at the same time training data is first seen with training labels obtained months later. We find that using training labels obtained well after samples appear, and thus unavailable in practice for current training data, inflates measured detection by almost 20 percentage points. We release our cluster-based implementation, as well as a list of all hashes in our evaluation and 3% of our entire dataset.