# On the Effectiveness of Adversarial Samples against Ensemble Learning-based Windows PE Malware Detectors

arXiv: https://arxiv.org/abs/2309.13841 · HTML: https://mlsanity.com/paper/2309.13841/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 55 references that are in the map.
- First posted on arXiv 25 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Adversarial Training and Robustness for Multiple Perturbations](https://mlsanity.com/paper/1904.13000/) (2019)
- [Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN](https://mlsanity.com/paper/1702.05983/) (2017)
- [Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning](https://mlsanity.com/paper/1801.08917/) (2018)
- [Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement](https://mlsanity.com/paper/2203.08008/) (2022)
- [Misleading Authorship Attribution of Source Code using Adversarial Learning](https://mlsanity.com/paper/1905.12386/) (2019)
- [A Survey of Machine Learning Methods and Challenges for Windows Malware Classification](https://mlsanity.com/paper/2006.09271/) (2020)
- [Adversarial Deep Ensemble: Evasion Attacks and Defenses for Malware Detection](https://mlsanity.com/paper/2006.16545/) (2020)
- [Malware Makeover: Breaking ML-based Static Analysis by Modifying Executable Bytes](https://mlsanity.com/paper/1912.09064/) (2019)
- [Deep Graph Matching and Searching for Semantic Code Retrieval](https://mlsanity.com/paper/2010.12908/) (2020)
- [AVClass2: Massive Malware Tag Extraction from AV Labels](https://mlsanity.com/paper/2006.10615/) (2020)
- [Adversarial Attacks against Windows PE Malware Detection: A Survey of the State-of-the-Art](https://mlsanity.com/paper/2112.12310/) (2021)
- [Machine Learning (In) Security: A Stream of Problems](https://mlsanity.com/paper/2010.16045/) (2020)
- [MalFox: Camouflaged Adversarial Malware Example Generation Based on Conv-GANs Against Black-Box Detectors](https://mlsanity.com/paper/2011.01509/) (2020)
- [ATMPA: Attacking Machine Learning-based Malware Visualization Detection Methods via Adversarial Examples](https://mlsanity.com/paper/1808.01546/) (2018)
- [Practical Attacks on Machine Learning: A Case Study on Adversarial Windows Malware](https://mlsanity.com/paper/2207.05548/) (2022)

## Abstract (arXiv)

Recently, there has been a growing focus and interest in applying machine learning (ML) to the field of cybersecurity, particularly in malware detection and prevention. Several research works on malware analysis have been proposed, offering promising results for both academic and practical applications. In these works, the use of Generative Adversarial Networks (GANs) or Reinforcement Learning (RL) can aid malware creators in crafting metamorphic malware that evades antivirus software. In this study, we propose a mutation system to counteract ensemble learning-based detectors by combining GANs and an RL model, overcoming the limitations of the MalGAN model. Our proposed FeaGAN model is built based on MalGAN by incorporating an RL model called the Deep Q-network anti-malware Engines Attacking Framework (DQEAF). The RL model addresses three key challenges in performing adversarial attacks on Windows Portable Executable malware, including format preservation, executability preservation, and maliciousness preservation. In the FeaGAN model, ensemble learning is utilized to enhance the malware detector's evasion ability, with the generated adversarial patterns. The experimental results demonstrate that 100% of the selected mutant samples preserve the format of executable files, while certain successes in both executability preservation and maliciousness preservation are achieved, reaching a stable success rate.