# Interpretable Summaries of Black Box Incident Triaging with Subgroup Discovery

arXiv: https://arxiv.org/abs/2108.03013 · HTML: https://mlsanity.com/paper/2108.03013/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 6 Aug 2021.

## Builds on (most cited first, 12 of 12)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [Global Explanations of Neural Networks: Mapping the Landscape of Predictions](https://mlsanity.com/paper/1902.02384/) (2019)
- [Black Box Explanation by Learning Image Exemplars in the Latent Feature Space](https://mlsanity.com/paper/2002.03746/) (2020)
- [Making Tree Ensembles Interpretable](https://mlsanity.com/paper/1606.05390/) (2016)
- [DeepTriage: Automated Transfer Assistance for Incidents in Cloud Services](https://mlsanity.com/paper/2012.03665/) (2020)
- [Fast Outage Analysis of Large-scale Production Clouds with Service Correlation Mining](https://mlsanity.com/paper/2103.03649/) (2021)

## Built on by (most cited first, 4 of 4)

- [Explainable Predictive Maintenance: A Survey of Current Methods, Challenges and Opportunities](https://mlsanity.com/paper/2401.07871/) (2024)
- [On-Premise AIOps Infrastructure for a Software Editor SME: An Experience Report](https://mlsanity.com/paper/2308.11225/) (2023)
- [How Can Subgroup Discovery Help AIOps?](https://mlsanity.com/paper/2109.04909/) (2021)
- [Exploiting Formal Concept Analysis for Data Modeling in Data Lakes](https://mlsanity.com/paper/2408.13265/) (2024)

## Abstract (arXiv)

The need of predictive maintenance comes with an increasing number of incidents reported by monitoring systems and equipment/software users. In the front line, on-call engineers (OCEs) have to quickly assess the degree of severity of an incident and decide which service to contact for corrective actions. To automate these decisions, several predictive models have been proposed, but the most efficient models are opaque (say, black box), strongly limiting their adoption. In this paper, we propose an efficient black box model based on 170K incidents reported to our company over the last 7 years and emphasize on the need of automating triage when incidents are massively reported on thousands of servers running our product, an ERP. Recent developments in eXplainable Artificial Intelligence (XAI) help in providing global explanations to the model, but also, and most importantly, with local explanations for each model prediction/outcome. Sadly, providing a human with an explanation for each outcome is not conceivable when dealing with an important number of daily predictions. To address this problem, we propose an original data-mining method rooted in Subgroup Discovery, a pattern mining technique with the natural ability to group objects that share similar explanations of their black box predictions and provide a description for each group. We evaluate this approach and present our preliminary results which give us good hope towards an effective OCE's adoption. We believe that this approach provides a new way to address the problem of model agnostic outcome explanation.