# Improving Botnet Detection with Recurrent Neural Network and Transfer Learning

arXiv: https://arxiv.org/abs/2104.12602 · HTML: https://mlsanity.com/paper/2104.12602/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 41 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 9 of 9)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music](https://mlsanity.com/paper/1803.05428/) (2018)
- [Anomaly Detection using One-Class Neural Networks](https://mlsanity.com/paper/1802.06360/) (2018)
- [GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection](https://mlsanity.com/paper/1903.06661/) (2019)
- [Transfer Learning for Image-Based Malware Classification](https://mlsanity.com/paper/1903.11551/) (2019)
- [Tracking Temporal Evolution of Network Activity for Botnet Detection](https://mlsanity.com/paper/1908.03443/) (2019)
- [On Designing Machine Learning Models for Malicious Network Traffic Classification](https://mlsanity.com/paper/1907.04846/) (2019)

## Abstract (arXiv)

Botnet detection is a critical step in stopping the spread of botnets and preventing malicious activities. However, reliable detection is still a challenging task, due to a wide variety of botnets involving ever-increasing types of devices and attack vectors. Recent approaches employing machine learning (ML) showed improved performance than earlier ones, but these ML- based approaches still have significant limitations. For example, most ML approaches can not incorporate sequential pattern analysis techniques key to detect some classes of botnets. Another common shortcoming of ML-based approaches is the need to retrain neural networks in order to detect the evolving botnets; however, the training process is time-consuming and requires significant efforts to label the training data. For fast-evolving botnets, it might take too long to create sufficient training samples before the botnets have changed again. To address these challenges, we propose a novel botnet detection method, built upon Recurrent Variational Autoencoder (RVAE) that effectively captures sequential characteristics of botnet activities. In the experiment, this semi-supervised learning method achieves better detection accuracy than similar learning methods, especially on hard to detect classes. Additionally, we devise a transfer learning framework to learn from a well-curated source data set and transfer the knowledge to a target problem domain not seen before. Tests show that the true-positive rate (TPR) with transfer learning is higher than the RVAE semi-supervised learning method trained using the target data set (91.8% vs. 68.3%).