# TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time

arXiv: https://arxiv.org/abs/1807.07838 · HTML: https://mlsanity.com/paper/1807.07838/

## Key facts

- Cited by 108 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2018.

## Built on by (most cited first, 30 of 108)

- [Continuous Learning for Android Malware Detection](https://mlsanity.com/paper/2302.04332/) (2023)
- [The Role of Machine Learning in Cybersecurity](https://mlsanity.com/paper/2206.09707/) (2022)
- [Deep Learning for Android Malware Defenses: a Systematic Literature Review](https://mlsanity.com/paper/2103.05292/) (2021)
- [LAMD: Context-driven Android Malware Detection and Classification with LLMs](https://mlsanity.com/paper/2502.13055/) (2025)
- [Adversarial Robustness with Non-uniform Perturbations](https://mlsanity.com/paper/2102.12002/) (2021)
- [MOTIF: A Large Malware Reference Dataset with Ground Truth Family Labels](https://mlsanity.com/paper/2111.15031/) (2021)
- [SoK: The Impact of Unlabelled Data in Cyberthreat Detection](https://mlsanity.com/paper/2205.08944/) (2022)
- [Learning Security Classifiers with Verified Global Robustness Properties](https://mlsanity.com/paper/2105.11363/) (2021)
- [Jigsaw Puzzle: Selective Backdoor Attack to Subvert Malware Classifiers](https://mlsanity.com/paper/2202.05470/) (2022)
- [EvadeDroid: A Practical Evasion Attack on Machine Learning for Black-box Android Malware Detection](https://mlsanity.com/paper/2110.03301/) (2021)
- [SoK: Pragmatic Assessment of Machine Learning for Network Intrusion Detection](https://mlsanity.com/paper/2305.00550/) (2023)
- [Decoding the Secrets of Machine Learning in Malware Classification: A Deep Dive into Datasets, Feature Extraction, and Model Performance](https://mlsanity.com/paper/2307.14657/) (2023)
- [On Benchmarking Code LLMs for Android Malware Analysis](https://mlsanity.com/paper/2504.00694/) (2025)
- [A Survey on Malware Detection with Graph Representation Learning](https://mlsanity.com/paper/2303.16004/) (2023)
- [Efficient Query-Based Attack against ML-Based Android Malware Detection under Zero Knowledge Setting](https://mlsanity.com/paper/2309.01866/) (2023)
- [DexRay: A Simple, yet Effective Deep Learning Approach to Android Malware Detection based on Image Representation of Bytecode](https://mlsanity.com/paper/2109.03326/) (2021)
- [Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey](https://mlsanity.com/paper/2310.17903/) (2023)
- [TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time (Extended Version)](https://mlsanity.com/paper/2402.01359/) (2024)
- [When AI Meets the Web: Prompt Injection Risks in Third-Party AI Chatbot Plugins](https://mlsanity.com/paper/2511.05797/) (2025)
- [MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection](https://mlsanity.com/paper/2302.10739/) (2023)
- [DRMD: Deep Reinforcement Learning for Malware Detection under Concept Drift](https://mlsanity.com/paper/2508.18839/) (2025)
- [Fast & Furious: Modelling Malware Detection as Evolving Data Streams](https://mlsanity.com/paper/2205.12311/) (2022)
- [Towards a Fair Comparison and Realistic Evaluation Framework of Android Malware Detectors based on Static Analysis and Machine Learning](https://mlsanity.com/paper/2205.12569/) (2022)
- [DexBERT: Effective, Task-Agnostic and Fine-grained Representation Learning of Android Bytecode](https://mlsanity.com/paper/2212.05976/) (2022)
- [LAMDA: A Longitudinal Android Malware Benchmark for Concept Drift Analysis](https://mlsanity.com/paper/2505.18551/) (2025)
- [Realizable Universal Adversarial Perturbations for Malware](https://mlsanity.com/paper/2102.06747/) (2021)
- [Can We Leverage Predictive Uncertainty to Detect Dataset Shift and Adversarial Examples in Android Malware Detection?](https://mlsanity.com/paper/2109.09654/) (2021)
- [PhishSim: Aiding Phishing Website Detection with a Feature-Free Tool](https://mlsanity.com/paper/2207.10801/) (2022)
- [Your Attack Is Too DUMB: Formalizing Attacker Scenarios for Adversarial Transferability](https://mlsanity.com/paper/2306.15363/) (2023)
- [Efficient Concept Drift Handling for Batch Android Malware Detection Models](https://mlsanity.com/paper/2309.09807/) (2023)

## Abstract (arXiv)

Is Android malware classification a solved problem? Published F1 scores of up to 0.99 appear to leave very little room for improvement. In this paper, we argue that results are commonly inflated due to two pervasive sources of experimental bias: "spatial bias" caused by distributions of training and testing data that are not representative of a real-world deployment; and "temporal bias" caused by incorrect time splits of training and testing sets, leading to impossible configurations. We propose a set of space and time constraints for experiment design that eliminates both sources of bias. We introduce a new metric that summarizes the expected robustness of a classifier in a real-world setting, and we present an algorithm to tune its performance. Finally, we demonstrate how this allows us to evaluate mitigation strategies for time decay such as active learning. We have implemented our solutions in TESSERACT, an open source evaluation framework for comparing malware classifiers in a realistic setting. We used TESSERACT to evaluate three Android malware classifiers from the literature on a dataset of 129K applications spanning over three years. Our evaluation confirms that earlier published results are biased, while also revealing counter-intuitive performance and showing that appropriate tuning can lead to significant improvements.