# DRLDO: A novel DRL based De-ObfuscationSystem for Defense against Metamorphic Malware

arXiv: https://arxiv.org/abs/2102.00898 · HTML: https://mlsanity.com/paper/2102.00898/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 1 Feb 2021.

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN](https://mlsanity.com/paper/1702.05983/) (2017)
- [Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning](https://mlsanity.com/paper/1801.08917/) (2018)
- [Adversarial Perturbations Against Deep Neural Networks for Malware Classification](https://mlsanity.com/paper/1606.04435/) (2016)
- [IDSGAN: Generative Adversarial Networks for Attack Generation against Intrusion Detection](https://mlsanity.com/paper/1809.02077/) (2018)
- [Malware Detection using Machine Learning and Deep Learning](https://mlsanity.com/paper/1904.02441/) (2019)
- [Comparison of Deep Learning and the Classical Machine Learning Algorithm for the Malware Detection](https://mlsanity.com/paper/1809.05889/) (2018)
- [Android Malicious Application Classification Using Clustering](https://mlsanity.com/paper/1904.10142/) (2019)
- [An investigation of a deep learning based malware detection system](https://mlsanity.com/paper/1809.05888/) (2018)
- [Grouping the executables to detect malware with high accuracy](https://mlsanity.com/paper/1606.06908/) (2016)
- [Metamorphic Malware Detection Using Linear Discriminant Analysis and Graph Similarity](https://mlsanity.com/paper/1811.04304/) (2018)

## Built on by (most cited first, 2 of 2)

- [Deep Reinforcement Learning for Cybersecurity Threat Detection and Protection: A Review](https://mlsanity.com/paper/2206.02733/) (2022)
- [LSTM Hyper-Parameter Selection for Malware Detection: Interaction Effects and Hierarchical Selection Approach](https://mlsanity.com/paper/2109.11500/) (2021)

## Abstract (arXiv)

In this paper, we propose a novel mechanism to normalize metamorphic and obfuscated malware down at the opcode level and hence create an advanced metamorphic malware de-obfuscation and defense system. We name this system DRLDO, for Deep Reinforcement Learning based De-Obfuscator. With the inclusion of the DRLDO as a sub-component, an existing Intrusion Detection System could be augmented with defensive capabilities against 'zero-day' attacks from obfuscated and metamorphic variants of existing malware. This gains importance, not only because there exists no system to date that uses advanced DRL to intelligently and automatically normalize obfuscation down even to the opcode level, but also because the DRLDO system does not mandate any changes to the existing IDS. The DRLDO system does not even mandate the IDS' classifier to be retrained with any new dataset containing obfuscated samples. Hence DRLDO could be easily retrofitted into any existing IDS deployment. We designed, developed, and conducted experiments on the system to evaluate the same against multiple-simultaneous attacks from obfuscations generated from malware samples from a standardized dataset that contains multiple generations of malware. Experimental results prove that DRLDO was able to successfully make the otherwise un-detectable obfuscated variants of the malware detectable by an existing pre-trained malware classifier. The detection probability was raised well above the cut-off mark to 0.6 for the classifier to detect the obfuscated malware unambiguously. Further, the de-obfuscated variants generated by DRLDO achieved a very high correlation (of 0.99) with the base malware. This observation validates that the DRLDO system is actually learning to de-obfuscate and not exploiting a trivial trick.