# Post-hoc explanation of black-box classifiers using confident itemsets

arXiv: https://arxiv.org/abs/2005.01992 · HTML: https://mlsanity.com/paper/2005.01992/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2020.

## Built on by (most cited first, 21 of 21)

- [Evaluating the Robustness of Neural Language Models to Input Perturbations](https://mlsanity.com/paper/2108.12237/) (2021)
- [Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey](https://mlsanity.com/paper/2104.00950/) (2021)
- [Calibrated Explanations: with Uncertainty Information and Counterfactuals](https://mlsanity.com/paper/2305.02305/) (2023)
- [A Survey on Explainable Artificial Intelligence for Cybersecurity](https://mlsanity.com/paper/2303.12942/) (2023)
- [A Meta Survey of Quality Evaluation Criteria in Explanation Methods](https://mlsanity.com/paper/2203.13929/) (2022)
- [Improving the robustness and accuracy of biomedical language models through adversarial training](https://mlsanity.com/paper/2111.08529/) (2021)
- [Model-agnostic explainable artificial intelligence for object detection in image data](https://mlsanity.com/paper/2303.17249/) (2023)
- [Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets](https://mlsanity.com/paper/2201.07724/) (2022)
- [Efficient Learning of Interpretable Classification Rules](https://mlsanity.com/paper/2205.06936/) (2022)
- [Calibrated Explanations for Regression](https://mlsanity.com/paper/2308.16245/) (2023)
- [Deep Learning, Natural Language Processing, and Explainable Artificial Intelligence in the Biomedical Domain](https://mlsanity.com/paper/2202.12678/) (2022)
- [Developing Guidelines for Functionally-Grounded Evaluation of Explainable Artificial Intelligence using Tabular Data](https://mlsanity.com/paper/2410.12803/) (2024)
- [ESC-Rules: Explainable, Semantically Constrained Rule Sets](https://mlsanity.com/paper/2208.12523/) (2022)
- [CLIMAX: An exploration of Classifier-Based Contrastive Explanations](https://mlsanity.com/paper/2307.00680/) (2023)
- [On the Definition of Appropriate Trust and the Tools that Come with it](https://mlsanity.com/paper/2309.11937/) (2023)
- [Fast Calibrated Explanations: Efficient and Uncertainty-Aware Explanations for Machine Learning Models](https://mlsanity.com/paper/2410.21129/) (2024)
- [Refining Visual Artifacts in Diffusion Models via Explainable AI-based Flaw Activation Maps](https://mlsanity.com/paper/2512.08774/) (2025)
- [AED: An black-box NLP classifier model attacker](https://mlsanity.com/paper/2112.11660/) (2021)
- [Causality-Aware Local Interpretable Model-Agnostic Explanations](https://mlsanity.com/paper/2212.05256/) (2022)
- [Analyzing Factors Influencing Driver Willingness to Accept Advanced Driver Assistance Systems](https://mlsanity.com/paper/2502.16688/) (2025)
- [Who Benefits from AI Explanations? Towards Accessible and Interpretable Systems](https://mlsanity.com/paper/2508.10806/) (2025)

## Abstract (arXiv)

Black-box Artificial Intelligence (AI) methods, e.g. deep neural networks, have been widely utilized to build predictive models that can extract complex relationships in a dataset and make predictions for new unseen data records. However, it is difficult to trust decisions made by such methods since their inner working and decision logic is hidden from the user. Explainable Artificial Intelligence (XAI) refers to systems that try to explain how a black-box AI model produces its outcomes. Post-hoc XAI methods approximate the behavior of a black-box by extracting relationships between feature values and the predictions. Perturbation-based and decision set methods are among commonly used post-hoc XAI systems. The former explanators rely on random perturbations of data records to build local or global linear models that explain individual predictions or the whole model. The latter explanators use those feature values that appear more frequently to construct a set of decision rules that produces the same outcomes as the target black-box. However, these two classes of XAI methods have some limitations. Random perturbations do not take into account the distribution of feature values in different subspaces, leading to misleading approximations. Decision sets only pay attention to frequent feature values and miss many important correlations between features and class labels that appear less frequently but accurately represent decision boundaries of the model. In this paper, we address the above challenges by proposing an explanation method named Confident Itemsets Explanation (CIE). We introduce confident itemsets, a set of feature values that are highly correlated to a specific class label. CIE utilizes confident itemsets to discretize the whole decision space of a model to smaller subspaces.