# Towards interpreting ML-based automated malware detection models: a survey

arXiv: https://arxiv.org/abs/2101.06232 · HTML: https://mlsanity.com/paper/2101.06232/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 152 references that are in the map.
- First posted on arXiv 15 Jan 2021.

## Builds on (most cited first, 30 of 35)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection](https://mlsanity.com/paper/2004.02015/) (2020)
- [Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach](https://mlsanity.com/paper/2006.04628/) (2020)
- [Explainable Neural Networks based on Additive Index Models](https://mlsanity.com/paper/1806.01933/) (2018)
- [Towards Explaining Anomalies: A Deep Taylor Decomposition of One-Class Models](https://mlsanity.com/paper/1805.06230/) (2018)
- [Interpretable Random Forests via Rule Extraction](https://mlsanity.com/paper/2004.14841/) (2020)
- [Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection](https://mlsanity.com/paper/1803.04967/) (2018)
- [Domain aware medical image classifier interpretation by counterfactual impact analysis](https://mlsanity.com/paper/2007.06312/) (2020)
- [Why an Android App is Classified as Malware? Towards Malware Classification Interpretation](https://mlsanity.com/paper/2004.11516/) (2020)
- [R2-D2: ColoR-inspired Convolutional NeuRal Network (CNN)-based AndroiD Malware Detections](https://mlsanity.com/paper/1705.04448/) (2017)
- [Early Stage Malware Prediction Using Recurrent Neural Networks](https://mlsanity.com/paper/1708.03513/) (2017)
- [Adversarial TCAV -- Robust and Effective Interpretation of Intermediate Layers in Neural Networks](https://mlsanity.com/paper/2002.03549/) (2020)
- [Local Interpretable Model-agnostic Explanations of Bayesian Predictive Models via Kullback-Leibler Projections](https://mlsanity.com/paper/1810.02678/) (2018)
- [Adversarial Infidelity Learning for Model Interpretation](https://mlsanity.com/paper/2006.05379/) (2020)
- [Similarity-based Android Malware Detection Using Hamming Distance of Static Binary Features](https://mlsanity.com/paper/1908.05759/) (2019)
- [MAB-Malware: A Reinforcement Learning Framework for Attacking Static Malware Classifiers](https://mlsanity.com/paper/2003.03100/) (2020)
- [Deep Transfer Learning for Static Malware Classification](https://mlsanity.com/paper/1812.07606/) (2018)
- [Prototype-based interpretation of the functionality of neurons in winner-take-all neural networks](https://mlsanity.com/paper/2008.08750/) (2020)
- [Explaining Black-box Android Malware Detection](https://mlsanity.com/paper/1803.03544/) (2018)
- [Interpreting and Understanding Graph Convolutional Neural Network using Gradient-based Attribution Method](https://mlsanity.com/paper/1903.03768/) (2019)

## Built on by (most cited first, 1 of 1)

- [Explainable Artificial Intelligence (XAI) for Malware Analysis: A Survey of Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2409.13723/) (2024)

## Abstract (arXiv)

Malware is being increasingly threatening and malware detectors based on traditional signature-based analysis are no longer suitable for current malware detection. Recently, the models based on machine learning (ML) are developed for predicting unknown malware variants and saving human strength. However, most of the existing ML models are black-box, which made their pre-diction results undependable, and therefore need further interpretation in order to be effectively deployed in the wild. This paper aims to examine and categorize the existing researches on ML-based malware detector interpretability. We first give a detailed comparison over the previous work on common ML model inter-pretability in groups after introducing the principles, attributes, evaluation indi-cators and taxonomy of common ML interpretability. Then we investigate the interpretation methods towards malware detection, by addressing the importance of interpreting malware detectors, challenges faced by this field, solutions for migitating these challenges, and a new taxonomy for classifying all the state-of-the-art malware detection interpretability work in recent years. The highlight of our survey is providing a new taxonomy towards malware detection interpreta-tion methods based on the common taxonomy summarized by previous re-searches in the common field. In addition, we are the first to evaluate the state-of-the-art approaches by interpretation method attributes to generate the final score so as to give insight to quantifying the interpretability. By concluding the results of the recent researches, we hope our work can provide suggestions for researchers who are interested in the interpretability on ML-based malware de-tection models.