# On the Importance of Domain-specific Explanations in AI-based Cybersecurity Systems (Technical Report)

arXiv: https://arxiv.org/abs/2108.02006 · HTML: https://mlsanity.com/paper/2108.02006/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations](https://mlsanity.com/paper/1907.09294/) (2019)
- [The Rise of Social Bots](https://mlsanity.com/paper/1407.5225/) (2014)
- [Visualizing Deep Networks by Optimizing with Integrated Gradients](https://mlsanity.com/paper/1905.00954/) (2019)
- [Human-in-the-loop Artificial Intelligence](https://mlsanity.com/paper/1710.08191/) (2017)
- [Explainable Security](https://mlsanity.com/paper/1807.04178/) (2018)
- [Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation](https://mlsanity.com/paper/2101.06930/) (2021)
- [Darknet and Deepnet Mining for Proactive Cybersecurity Threat Intelligence](https://mlsanity.com/paper/1607.08583/) (2016)
- [Human-in-the-Loop SLAM](https://mlsanity.com/paper/1711.08566/) (2017)
- [Belief Revision in Structured Probabilistic Argumentation](https://mlsanity.com/paper/1401.1475/) (2014)

## Built on by (most cited first, 7 of 7)

- [Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy](https://mlsanity.com/paper/2402.04247/) (2024)
- [SoK: Explainable Machine Learning for Computer Security Applications](https://mlsanity.com/paper/2208.10605/) (2022)
- [Detection, Explanation and Filtering of Cyber Attacks Combining Symbolic and Sub-Symbolic Methods](https://mlsanity.com/paper/2212.13991/) (2022)
- [Trustworthy and Responsible AI for Human-Centric Autonomous Decision-Making Systems](https://mlsanity.com/paper/2408.15550/) (2024)
- [A Sim2Real Approach for Identifying Task-Relevant Properties in Interpretable Machine Learning](https://mlsanity.com/paper/2406.00116/) (2024)
- [Explainable Artificial Intelligence and Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2303.01259/) (2023)
- [Towards Explainable Meta-Learning for DDoS Detection](https://mlsanity.com/paper/2204.02255/) (2022)

## Abstract (arXiv)

With the availability of large datasets and ever-increasing computing power, there has been a growing use of data-driven artificial intelligence systems, which have shown their potential for successful application in diverse areas. However, many of these systems are not able to provide information about the rationale behind their decisions to their users. Lack of understanding of such decisions can be a major drawback, especially in critical domains such as those related to cybersecurity. In light of this problem, in this paper we make three contributions: (i) proposal and discussion of desiderata for the explanation of outputs generated by AI-based cybersecurity systems; (ii) a comparative analysis of approaches in the literature on Explainable Artificial Intelligence (XAI) under the lens of both our desiderata and further dimensions that are typically used for examining XAI approaches; and (iii) a general architecture that can serve as a roadmap for guiding research efforts towards the development of explainable AI-based cybersecurity systems -- at its core, this roadmap proposes combinations of several research lines in a novel way towards tackling the unique challenges that arise in this context.