# Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box arXiv: https://arxiv.org/abs/2203.02399 ยท HTML: https://mlsanity.com/paper/2203.02399/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Builds on 44 of its 100 references that are in the map. - First posted on arXiv 4 Mar 2022. ## Builds on (most cited first, 30 of 44) - [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017) - ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016) - [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017) - [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017) - [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017) - [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017) - [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018) - [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016) - [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019) - [Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives](https://mlsanity.com/paper/1802.07623/) (2018) - [Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey](https://mlsanity.com/paper/2006.11371/) (2020) - [Bias and Debias in Recommender System: A Survey and Future Directions](https://mlsanity.com/paper/2010.03240/) (2020) - [FACE: Feasible and Actionable Counterfactual Explanations](https://mlsanity.com/paper/1909.09369/) (2019) - [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020) - [Interpretable Counterfactual Explanations Guided by Prototypes](https://mlsanity.com/paper/1907.02584/) (2019) - [Model-Agnostic Counterfactual Explanations for Consequential Decisions](https://mlsanity.com/paper/1905.11190/) (2019) - [Multi-Objective Counterfactual Explanations](https://mlsanity.com/paper/2004.11165/) (2020) - [Learning Model-Agnostic Counterfactual Explanations for Tabular Data](https://mlsanity.com/paper/1910.09398/) (2019) - [Algorithmic recourse under imperfect causal knowledge: a probabilistic approach](https://mlsanity.com/paper/2006.06831/) (2020) - [Benchmarking and Survey of Explanation Methods for Black Box Models](https://mlsanity.com/paper/2102.13076/) (2021) - [A survey of algorithmic recourse: definitions, formulations, solutions, and prospects](https://mlsanity.com/paper/2010.04050/) (2020) - [The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons](https://mlsanity.com/paper/1912.04930/) (2019) - [Efficient Search for Diverse Coherent Explanations](https://mlsanity.com/paper/1901.04909/) (2019) - [Principles and Practice of Explainable Machine Learning](https://mlsanity.com/paper/2009.11698/) (2020) - [The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations](https://mlsanity.com/paper/1907.09294/) (2019) - [If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques](https://mlsanity.com/paper/2103.01035/) (2021) - [Good Counterfactuals and Where to Find Them: A Case-Based Technique for Generating Counterfactuals for Explainable AI (XAI)](https://mlsanity.com/paper/2005.13997/) (2020) - [Towards Unifying Feature Attribution and Counterfactual Explanations: Different Means to the Same End](https://mlsanity.com/paper/2011.04917/) (2020) - [Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications](https://mlsanity.com/paper/2103.04244/) (2021) - [Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts](https://mlsanity.com/paper/2201.10295/) (2022) ## Built on by (most cited first, 7 of 7) - [The role of causality in explainable artificial intelligence](https://mlsanity.com/paper/2309.09901/) (2023) - [Interpretable Clustering: A Survey](https://mlsanity.com/paper/2409.00743/) (2024) - [Enhancing Counterfactual Explanation Search with Diffusion Distance and Directional Coherence](https://mlsanity.com/paper/2404.12810/) (2024) - [Towards a Comprehensive Human-Centred Evaluation Framework for Explainable AI](https://mlsanity.com/paper/2308.06274/) (2023) - [Tabular Diffusion based Actionable Counterfactual Explanations for Network Intrusion Detection](https://mlsanity.com/paper/2507.17161/) (2025) - [Weakly Supervised Pixel-Level Annotation with Visual Interpretability](https://mlsanity.com/paper/2502.17824/) (2025) - [KTCF: Actionable Recourse in Knowledge Tracing via Counterfactual Explanations for Education](https://mlsanity.com/paper/2601.09156/) (2026) ## Abstract (arXiv) This study investigates the impact of machine learning models on the generation of counterfactual explanations by conducting a benchmark evaluation over three different types of models: a decision tree (fully transparent, interpretable, white-box model), a random forest (semi-interpretable, grey-box model), and a neural network (fully opaque, black-box model). We tested the counterfactual generation process using four algorithms (DiCE, WatcherCF, prototype, and GrowingSpheresCF) in the literature in 25 different datasets. Our findings indicate that: (1) Different machine learning models have little impact on the generation of counterfactual explanations; (2) Counterfactual algorithms based uniquely on proximity loss functions are not actionable and will not provide meaningful explanations; (3) One cannot have meaningful evaluation results without guaranteeing plausibility in the counterfactual generation. Algorithms that do not consider plausibility in their internal mechanisms will lead to biased and unreliable conclusions if evaluated with the current state-of-the-art metrics; (4) A counterfactual inspection analysis is strongly recommended to ensure a robust examination of counterfactual explanations and the potential identification of biases.