# An Interpretable Probabilistic Approach for Demystifying Black-box Predictive Models

arXiv: https://arxiv.org/abs/2007.10668 · HTML: https://mlsanity.com/paper/2007.10668/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2020.

## Built on by (most cited first, 7 of 7)

- [Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications](https://mlsanity.com/paper/2103.04244/) (2021)
- [DiCE4EL: Interpreting Process Predictions using a Milestone-Aware Counterfactual Approach](https://mlsanity.com/paper/2107.08697/) (2021)
- [Developing a Fidelity Evaluation Approach for Interpretable Machine Learning](https://mlsanity.com/paper/2106.08492/) (2021)
- [Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box](https://mlsanity.com/paper/2203.02399/) (2022)
- [Generative Perturbation Analysis for Probabilistic Black-Box Anomaly Attribution](https://mlsanity.com/paper/2308.04708/) (2023)
- [MDF-Net for abnormality detection by fusing X-rays with clinical data](https://mlsanity.com/paper/2302.13390/) (2023)
- [A Neuro-Symbolic Explainer for Rare Events: A Case Study on Predictive Maintenance](https://mlsanity.com/paper/2404.14455/) (2024)

## Abstract (arXiv)

The use of sophisticated machine learning models for critical decision making is faced with a challenge that these models are often applied as a "black-box". This has led to an increased interest in interpretable machine learning, where post hoc interpretation presents a useful mechanism for generating interpretations of complex learning models. In this paper, we propose a novel approach underpinned by an extended framework of Bayesian networks for generating post hoc interpretations of a black-box predictive model. The framework supports extracting a Bayesian network as an approximation of the black-box model for a specific prediction. Compared to the existing post hoc interpretation methods, the contribution of our approach is three-fold. Firstly, the extracted Bayesian network, as a probabilistic graphical model, can provide interpretations about not only what input features but also why these features contributed to a prediction. Secondly, for complex decision problems with many features, a Markov blanket can be generated from the extracted Bayesian network to provide interpretations with a focused view on those input features that directly contributed to a prediction. Thirdly, the extracted Bayesian network enables the identification of four different rules which can inform the decision-maker about the confidence level in a prediction, thus helping the decision-maker assess the reliability of predictions learned by a black-box model. We implemented the proposed approach, applied it in the context of two well-known public datasets and analysed the results, which are made available in an open-source repository.