# Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities

arXiv: https://arxiv.org/abs/2207.06236 · HTML: https://mlsanity.com/paper/2207.06236/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 175 references that are in the map.
- First posted on arXiv 13 Jul 2022.

## Builds on (most cited first, 30 of 50)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1605.01713/) (2016)
- [Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey](https://mlsanity.com/paper/2006.11371/) (2020)
- [Explaining NonLinear Classification Decisions with Deep Taylor Decomposition](https://mlsanity.com/paper/1512.02479/) (2015)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers](https://mlsanity.com/paper/1604.00825/) (2016)
- [One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques](https://mlsanity.com/paper/1909.03012/) (2019)
- [What do we need to build explainable AI systems for the medical domain?](https://mlsanity.com/paper/1712.09923/) (2017)
- [Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches](https://mlsanity.com/paper/1912.05100/) (2019)
- [Explaining Recurrent Neural Network Predictions in Sentiment Analysis](https://mlsanity.com/paper/1706.07206/) (2017)
- [Explainable Deep Few-shot Anomaly Detection with Deviation Networks](https://mlsanity.com/paper/2108.00462/) (2021)
- [Explainable k-Means and k-Medians Clustering](https://mlsanity.com/paper/2002.12538/) (2020)
- [Improving Question Answering by Commonsense-Based Pre-Training](https://mlsanity.com/paper/1809.03568/) (2018)
- [Explainable Artificial Intelligence Approaches: A Survey](https://mlsanity.com/paper/2101.09429/) (2021)
- [GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection](https://mlsanity.com/paper/1903.06661/) (2019)

## Built on by (most cited first, 25 of 25)

- [Towards Explainable Network Intrusion Detection using Large Language Models](https://mlsanity.com/paper/2408.04342/) (2024)
- [Explainable Predictive Maintenance: A Survey of Current Methods, Challenges and Opportunities](https://mlsanity.com/paper/2401.07871/) (2024)
- [Security Considerations in AI-Robotics: A Survey of Current Methods, Challenges, and Opportunities](https://mlsanity.com/paper/2310.08565/) (2023)
- [Impacts and Risk of Generative AI Technology on Cyber Defense](https://mlsanity.com/paper/2306.13033/) (2023)
- [Use of Graph Neural Networks in Aiding Defensive Cyber Operations](https://mlsanity.com/paper/2401.05680/) (2024)
- [Creating an Explainable Intrusion Detection System Using Self Organizing Maps](https://mlsanity.com/paper/2207.07465/) (2022)
- [TwinExplainer: Explaining Predictions of an Automotive Digital Twin](https://mlsanity.com/paper/2302.00152/) (2023)
- [A Temporal Anomaly Detection System for Vehicles utilizing Functional Working Groups and Sensor Channels](https://mlsanity.com/paper/2209.06828/) (2022)
- [eX-NIDS: A Framework for Explainable Network Intrusion Detection Leveraging Large Language Models](https://mlsanity.com/paper/2507.16241/) (2025)
- [A Survey on Privacy Attacks Against Digital Twin Systems in AI-Robotics](https://mlsanity.com/paper/2406.18812/) (2024)
- [Explainable Intrusion Detection Systems Using Competitive Learning Techniques](https://mlsanity.com/paper/2303.17387/) (2023)
- [Interpretable Anomaly-Based DDoS Detection in AI-RAN with XAI and LLMs](https://mlsanity.com/paper/2507.21193/) (2025)
- [Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities](https://mlsanity.com/paper/2509.06921/) (2025)
- [X-CBA: Explainability Aided CatBoosted Anomal-E for Intrusion Detection System](https://mlsanity.com/paper/2402.00839/) (2024)
- [A Comparative Analysis of DNN-based White-Box Explainable AI Methods in Network Security](https://mlsanity.com/paper/2501.07801/) (2025)
- [Robust Intrusion Detection System with Explainable Artificial Intelligence](https://mlsanity.com/paper/2503.05303/) (2025)
- [CAN-QA: A Question-Answering Benchmark for Reasoning over In-Vehicle CAN Traffic](https://mlsanity.com/paper/2604.24935/) (2026)
- [XFedHunter: An Explainable Federated Learning Framework for Advanced Persistent Threat Detection in SDN](https://mlsanity.com/paper/2309.08485/) (2023)
- [Eclectic Rule Extraction for Explainability of Deep Neural Network based Intrusion Detection Systems](https://mlsanity.com/paper/2401.10207/) (2024)
- [XAI-based Feature Selection for Improved Network Intrusion Detection Systems](https://mlsanity.com/paper/2410.10050/) (2024)
- [A Global Analysis of Cyber Threats to the Energy Sector: "Currents of Conflict" from a Geopolitical Perspective](https://mlsanity.com/paper/2509.22280/) (2025)
- [Spatio-Temporal Attention Graph Neural Network: Explaining Causalities With Attention](https://mlsanity.com/paper/2603.10676/) (2026)
- [XAI and Statistical Analysis for Reliable Intrusion Detection in the UAVIDS-2025 Dataset: From Tree to Hybrid and Tabular DNN Ensembles](https://mlsanity.com/paper/2605.13922/) (2026)
- [From Detection to Response: A Deep Learning and Retrieval-Augmented Generation Framework for Network Intrusion Mitigation](https://mlsanity.com/paper/2605.17960/) (2026)
- [Generative AI and Federated Learning for Intrusion Detection Systems: A Survey](https://mlsanity.com/paper/2607.01305/) (2026)

## Abstract (arXiv)

The application of Artificial Intelligence (AI) and Machine Learning (ML) to cybersecurity challenges has gained traction in industry and academia, partially as a result of widespread malware attacks on critical systems such as cloud infrastructures and government institutions. Intrusion Detection Systems (IDS), using some forms of AI, have received widespread adoption due to their ability to handle vast amounts of data with a high prediction accuracy. These systems are hosted in the organizational Cyber Security Operation Center (CSoC) as a defense tool to monitor and detect malicious network flow that would otherwise impact the Confidentiality, Integrity, and Availability (CIA). CSoC analysts rely on these systems to make decisions about the detected threats. However, IDSs designed using Deep Learning (DL) techniques are often treated as black box models and do not provide a justification for their predictions. This creates a barrier for CSoC analysts, as they are unable to improve their decisions based on the model's predictions. One solution to this problem is to design explainable IDS (X-IDS). This survey reviews the state-of-the-art in explainable AI (XAI) for IDS, its current challenges, and discusses how these challenges span to the design of an X-IDS. In particular, we discuss black box and white box approaches comprehensively. We also present the tradeoff between these approaches in terms of their performance and ability to produce explanations. Furthermore, we propose a generic architecture that considers human-in-the-loop which can be used as a guideline when designing an X-IDS. Research recommendations are given from three critical viewpoints: the need to define explainability for IDS, the need to create explanations tailored to various stakeholders, and the need to design metrics to evaluate explanations.