# Evaluating the robustness of adversarial defenses in malware detection systems

arXiv: https://arxiv.org/abs/2505.09342 · HTML: https://mlsanity.com/paper/2505.09342/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 14 May 2025.

## Builds on (most cited first, 25 of 25)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [On the (Statistical) Detection of Adversarial Examples](https://mlsanity.com/paper/1702.06280/) (2017)
- [Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN](https://mlsanity.com/paper/1702.05983/) (2017)
- [Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent](https://mlsanity.com/paper/2106.15023/) (2021)
- [Adversarial Deep Ensemble: Evasion Attacks and Defenses for Malware Detection](https://mlsanity.com/paper/2006.16545/) (2020)
- [Adversarial Deep Learning for Robust Detection of Binary Encoded Malware](https://mlsanity.com/paper/1801.02950/) (2018)
- [MaMaDroid: Detecting Android Malware by Building Markov Chains of Behavioral Models](https://mlsanity.com/paper/1612.04433/) (2016)
- [DLA: Dense-Layer-Analysis for Adversarial Example Detection](https://mlsanity.com/paper/1911.01921/) (2019)
- [Automated Poisoning Attacks and Defenses in Malware Detection Systems: An Adversarial Machine Learning Approach](https://mlsanity.com/paper/1706.04146/) (2017)
- [Android HIV: A Study of Repackaging Malware for Evading Machine-Learning Detection](https://mlsanity.com/paper/1808.04218/) (2018)
- [Deep Learning for Android Malware Defenses: a Systematic Literature Review](https://mlsanity.com/paper/2103.05292/) (2021)
- [EvadeDroid: A Practical Evasion Attack on Machine Learning for Black-box Android Malware Detection](https://mlsanity.com/paper/2110.03301/) (2021)
- [Black-box Adversarial Example Attack towards FCG Based Android Malware Detection under Incomplete Feature Information](https://mlsanity.com/paper/2303.08509/) (2023)
- [σ-zero: Gradient-based Optimization of ℓ0-norm Adversarial Examples](https://mlsanity.com/paper/2402.01879/) (2024)
- [A Framework for Enhancing Deep Neural Networks Against Adversarial Malware](https://mlsanity.com/paper/2004.07919/) (2020)
- [PAD: Towards Principled Adversarial Malware Detection Against Evasion Attacks](https://mlsanity.com/paper/2302.11328/) (2023)
- [HashTran-DNN: A Framework for Enhancing Robustness of Deep Neural Networks against Adversarial Malware Samples](https://mlsanity.com/paper/1809.06498/) (2018)
- [Unraveling the Key of Machine Learning-based Android Malware Detection](https://mlsanity.com/paper/2402.02953/) (2024)
- [MalPurifier: Enhancing Android Malware Detection with Adversarial Purification against Evasion Attacks](https://mlsanity.com/paper/2312.06423/) (2023)

## Abstract (arXiv)

Machine learning is a key tool for Android malware detection, effectively identifying malicious patterns in apps. However, ML-based detectors are vulnerable to evasion attacks, where small, crafted changes bypass detection. Despite progress in adversarial defenses, the lack of comprehensive evaluation frameworks in binary-constrained domains limits understanding of their robustness. We introduce two key contributions. First, Prioritized Binary Rounding, a technique to convert continuous perturbations into binary feature spaces while preserving high attack success and low perturbation size. Second, the sigma-binary attack, a novel adversarial method for binary domains, designed to achieve attack goals with minimal feature changes. Experiments on the Malscan dataset show that sigma-binary outperforms existing attacks and exposes key vulnerabilities in state-of-the-art defenses. Defenses equipped with adversary detectors, such as KDE, DLA, DNN+, and ICNN, exhibit significant brittleness, with attack success rates exceeding 90% using fewer than 10 feature modifications and reaching 100% with just 20. Adversarially trained defenses, including AT-rFGSM-k, AT-MaxMA, improves robustness under small budgets but remains vulnerable to unrestricted perturbations, with attack success rates of 99.45% and 96.62%, respectively. Although PAD-SMA demonstrates strong robustness against state-of-the-art gradient-based adversarial attacks by maintaining an attack success rate below 16.55%, the sigma-binary attack significantly outperforms these methods, achieving a 94.56% success rate under unrestricted perturbations. These findings highlight the critical need for precise method like sigma-binary to expose hidden vulnerabilities in existing defenses and support the development of more resilient malware detection systems.