# Keep Your Friends Close and Your Counterfactuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Setting

arXiv: https://arxiv.org/abs/2205.05515 · HTML: https://mlsanity.com/paper/2205.05515/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 74 references that are in the map.
- First posted on arXiv 11 May 2022.

## Builds on (most cited first, 23 of 23)

- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [FACE: Feasible and Actionable Counterfactual Explanations](https://mlsanity.com/paper/1909.09369/) (2019)
- [Interpretable Counterfactual Explanations Guided by Prototypes](https://mlsanity.com/paper/1907.02584/) (2019)
- [Local Rule-Based Explanations of Black Box Decision Systems](https://mlsanity.com/paper/1805.10820/) (2018)
- [Multi-Objective Counterfactual Explanations](https://mlsanity.com/paper/2004.11165/) (2020)
- [A survey of algorithmic recourse: definitions, formulations, solutions, and prospects](https://mlsanity.com/paper/2010.04050/) (2020)
- [If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques](https://mlsanity.com/paper/2103.01035/) (2021)
- [Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations](https://mlsanity.com/paper/1912.09024/) (2019)
- [Measurable Counterfactual Local Explanations for Any Classifier](https://mlsanity.com/paper/1908.03020/) (2019)
- [Evaluating Robustness of Counterfactual Explanations](https://mlsanity.com/paper/2103.02354/) (2021)
- [On the computation of counterfactual explanations -- A survey](https://mlsanity.com/paper/1911.07749/) (2019)
- [Convex Density Constraints for Computing Plausible Counterfactual Explanations](https://mlsanity.com/paper/2002.04862/) (2020)
- [GeCo: Quality Counterfactual Explanations in Real Time](https://mlsanity.com/paper/2101.01292/) (2021)
- [Issues with post-hoc counterfactual explanations: a discussion](https://mlsanity.com/paper/1906.04774/) (2019)
- [A Few Good Counterfactuals: Generating Interpretable, Plausible and Diverse Counterfactual Explanations](https://mlsanity.com/paper/2101.09056/) (2021)
- [A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data](https://mlsanity.com/paper/2107.04680/) (2021)
- [Let's Go to the Alien Zoo: Introducing an Experimental Framework to Study Usability of Counterfactual Explanations for Machine Learning](https://mlsanity.com/paper/2205.03398/) (2022)
- ["I know it when I see it". Visualization and Intuitive Interpretability](https://mlsanity.com/paper/1711.08042/) (2017)
- [The Role of Individual User Differences in Interpretable and Explainable Machine Learning Systems](https://mlsanity.com/paper/2009.06675/) (2020)

## Built on by (most cited first, 11 of 11)

- [Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations](https://mlsanity.com/paper/2210.11584/) (2022)
- [Even if Explanations: Prior Work, Desiderata & Benchmarks for Semi-Factual XAI](https://mlsanity.com/paper/2301.11970/) (2023)
- [Counterfactual Explanations for Misclassified Images: How Human and Machine Explanations Differ](https://mlsanity.com/paper/2212.08733/) (2022)
- ["Even if ..." -- Diverse Semifactual Explanations of Reject](https://mlsanity.com/paper/2207.01898/) (2022)
- [Explaining Groups of Instances Counterfactually for XAI: A Use Case, Algorithm and User Study for Group-Counterfactuals](https://mlsanity.com/paper/2303.09297/) (2023)
- [Generating Likely Counterfactuals Using Sum-Product Networks](https://mlsanity.com/paper/2401.14086/) (2024)
- [For Better or Worse: The Impact of Counterfactual Explanations' Directionality on User Behavior in xAI](https://mlsanity.com/paper/2306.07637/) (2023)
- [Reassessing Evaluation Functions in Algorithmic Recourse: An Empirical Study from a Human-Centered Perspective](https://mlsanity.com/paper/2405.14264/) (2024)
- [People Attribute Purpose to Autonomous Vehicles When Explaining Their Behavior: Insights from Cognitive Science for Explainable AI](https://mlsanity.com/paper/2403.08828/) (2024)
- [When Bias Backfires: The Modulatory Role of Counterfactual Explanations on the Adoption of Algorithmic Bias in XAI-Supported Human Decision-Making](https://mlsanity.com/paper/2505.14377/) (2025)
- [Do Metrics for Counterfactual Explanations Align with User Perception?](https://mlsanity.com/paper/2603.15607/) (2026)

## Abstract (arXiv)

Counterfactual explanations (CFEs) highlight what changes to a model's input would have changed its prediction in a particular way. CFEs have gained considerable traction as a psychologically grounded solution for explainable artificial intelligence (XAI). Recent innovations introduce the notion of computational plausibility for automatically generated CFEs, enhancing their robustness by exclusively creating plausible explanations. However, practical benefits of such a constraint on user experience and behavior is yet unclear. In this study, we evaluate objective and subjective usability of computationally plausible CFEs in an iterative learning design targeting novice users. We rely on a novel, game-like experimental design, revolving around an abstract scenario. Our results show that novice users actually benefit less from receiving computationally plausible rather than closest CFEs that produce minimal changes leading to the desired outcome. Responses in a post-game survey reveal no differences in terms of subjective user experience between both groups. Following the view of psychological plausibility as comparative similarity, this may be explained by the fact that users in the closest condition experience their CFEs as more psychologically plausible than the computationally plausible counterpart. In sum, our work highlights a little-considered divergence of definitions of computational plausibility and psychological plausibility, critically confirming the need to incorporate human behavior, preferences and mental models already at the design stages of XAI approaches. In the interest of reproducible research, all source code, acquired user data, and evaluation scripts of the current study are available: https://github.com/ukuhl/PlausibleAlienZoo