# TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time (Extended Version)

arXiv: https://arxiv.org/abs/2402.01359 · HTML: https://mlsanity.com/paper/2402.01359/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 78 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models](https://mlsanity.com/paper/1804.04637/) (2018)
- [TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time](https://mlsanity.com/paper/1807.07838/) (2018)
- [Dos and Don'ts of Machine Learning in Computer Security](https://mlsanity.com/paper/2010.09470/) (2020)
- [Transcending Transcend: Revisiting Malware Classification in the Presence of Concept Drift](https://mlsanity.com/paper/2010.03856/) (2020)
- [Yes, Machine Learning Can Be More Secure! A Case Study on Android Malware Detection](https://mlsanity.com/paper/1704.08996/) (2017)
- [Continuous Learning for Android Malware Detection](https://mlsanity.com/paper/2302.04332/) (2023)
- [MaMaDroid: Detecting Android Malware by Building Markov Chains of Behavioral Models](https://mlsanity.com/paper/1612.04433/) (2016)
- [Reviewer Integration and Performance Measurement for Malware Detection](https://mlsanity.com/paper/1510.07338/) (2015)
- [Adversarial Attacks against Windows PE Malware Detection: A Survey of the State-of-the-Art](https://mlsanity.com/paper/2112.12310/) (2021)
- [Benchmarking Crimes: An Emerging Threat in Systems Security](https://mlsanity.com/paper/1801.02381/) (2018)

## Built on by (most cited first, 9 of 9)

- [DRMD: Deep Reinforcement Learning for Malware Detection under Concept Drift](https://mlsanity.com/paper/2508.18839/) (2025)
- [SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity](https://mlsanity.com/paper/2602.08690/) (2026)
- [Understanding Concept Drift with Deprecated Permissions in Android Malware Detection](https://mlsanity.com/paper/2507.22231/) (2025)
- [Direction for Detection: A Survey of Automated Vulnerability Detection and all of its Pain Points](https://mlsanity.com/paper/2412.11194/) (2024)
- [MAWIFlow Benchmark: Realistic Flow-Based Evaluation for Network Intrusion Detection](https://mlsanity.com/paper/2506.17041/) (2025)
- [A Comprehensive Evaluation of Code Language Models for Security Patch Detection](https://mlsanity.com/paper/2605.13138/) (2026)
- [Malaika: Understanding Malware through Tri-Grounded Agentic Reasoning](https://mlsanity.com/paper/2607.09179/) (2026)
- [How Benchmarks and Evaluation Protocols Shape Conclusions in Provenance-Based Intrusion Detection](https://mlsanity.com/paper/2608.01454/) (2026)
- [REPLICANT: Learning Policies for Evading and Hardening Malware Detectors](https://mlsanity.com/paper/2608.28499/) (2026)

## Abstract (arXiv)

Machine learning (ML) plays a pivotal role in detecting malicious software. Despite the high F1-scores reported in numerous studies reaching upwards of 0.99, the issue is not completely solved. Malware detectors often experience performance decay due to constantly evolving operating systems and attack methods, which can render previously learned knowledge insufficient for accurate decision-making on new inputs. This paper argues that commonly reported results are inflated due to two pervasive sources of experimental bias in the detection task: spatial bias caused by data distributions that are not representative of a real-world deployment; and temporal bias caused by incorrect time splits of data, leading to unrealistic configurations. To address these biases, we introduce a set of constraints for fair experiment design, and propose a new metric, AUT, for classifier robustness in real-world settings. We additionally propose an algorithm designed to tune training data to enhance classifier performance. Finally, we present TESSERACT, an open-source framework for realistic classifier comparison. Our evaluation encompasses both traditional ML and deep learning methods, examining published works on an extensive Android dataset with 259,230 samples over a five-year span. Additionally, we conduct case studies in the Windows PE and PDF domains. Our findings identify the existence of biases in previous studies and reveal that significant performance enhancements are possible through appropriate, periodic tuning. We explore how mitigation strategies may support in achieving a more stable and better performance over time by employing multiple strategies to delay performance decay.