# Deep Q-Learning based Reinforcement Learning Approach for Network Intrusion Detection

arXiv: https://arxiv.org/abs/2111.13978 · HTML: https://mlsanity.com/paper/2111.13978/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 47 references that are in the map.
- First posted on arXiv 27 Nov 2021.

## Builds on (most cited first, 6 of 6)

- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation](https://mlsanity.com/paper/1802.07228/) (2018)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimization](https://mlsanity.com/paper/2006.01610/) (2020)
- [MSD-Kmeans: A Novel Algorithm for Efficient Detection of Global and Local Outliers](https://mlsanity.com/paper/1910.06588/) (2019)
- [Cyber Situation Awareness Monitoring and Proactive Response for Enterprises on the Cloud](https://mlsanity.com/paper/2009.01604/) (2020)

## Built on by (most cited first, 8 of 8)

- [IGRF-RFE: A Hybrid Feature Selection Method for MLP-based Network Intrusion Detection on UNSW-NB15 Dataset](https://mlsanity.com/paper/2203.16365/) (2022)
- [Discovering Command and Control Channels Using Reinforcement Learning](https://mlsanity.com/paper/2401.07154/) (2024)
- [DRL-GAN: A Hybrid Approach for Binary and Multiclass Network Intrusion Detection](https://mlsanity.com/paper/2301.03368/) (2023)
- [Dealing with uncertainty: balancing exploration and exploitation in deep recurrent reinforcement learning](https://mlsanity.com/paper/2310.08331/) (2023)
- [A Survey for Deep Reinforcement Learning Based Network Intrusion Detection](https://mlsanity.com/paper/2410.07612/) (2024)
- [Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications](https://mlsanity.com/paper/2505.19837/) (2025)
- [A Multi-Layer Cloud-IDS Pipeline with LLM and Adaptive Q-Learning Calibration](https://mlsanity.com/paper/2605.15889/) (2026)
- [Dueling Deep Q-Learning for Intrusion Detection](https://mlsanity.com/paper/2608.11291/) (2026)

## Abstract (arXiv)

The rise of the new generation of cyber threats demands more sophisticated and intelligent cyber defense solutions equipped with autonomous agents capable of learning to make decisions without the knowledge of human experts. Several reinforcement learning methods (e.g., Markov) for automated network intrusion tasks have been proposed in recent years. In this paper, we introduce a new generation of network intrusion detection methods that combines a Q-learning-based reinforcement learning with a deep-feed forward neural network method for network intrusion detection. Our proposed Deep Q-Learning (DQL) model provides an ongoing auto-learning capability for a network environment that can detect different types of network intrusions using an automated trial-error approach and continuously enhance its detection capabilities. We provide the details of fine-tuning different hyperparameters involved in the DQL model for more effective self-learning. According to our extensive experimental results based on the NSL-KDD dataset, we confirm that the lower discount factor which is set as 0.001 under 250 episodes of training yields the best performance results. Our experimental results also show that our proposed DQL is highly effective in detecting different intrusion classes and outperforms other similar machine learning approaches.