# For Better or Worse: The Impact of Counterfactual Explanations' Directionality on User Behavior in xAI

arXiv: https://arxiv.org/abs/2306.07637 · HTML: https://mlsanity.com/paper/2306.07637/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 69 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 23 of 23)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Actionable Recourse in Linear Classification](https://mlsanity.com/paper/1809.06514/) (2018)
- [Model-Agnostic Counterfactual Explanations for Consequential Decisions](https://mlsanity.com/paper/1905.11190/) (2019)
- [Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review](https://mlsanity.com/paper/2010.10596/) (2020)
- [The Shapley Value in Machine Learning](https://mlsanity.com/paper/2202.05594/) (2022)
- [Good Counterfactuals and Where to Find Them: A Case-Based Technique for Generating Counterfactuals for Explainable AI (XAI)](https://mlsanity.com/paper/2005.13997/) (2020)
- [Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations](https://mlsanity.com/paper/1912.09024/) (2019)
- [Interpretable Credit Application Predictions With Counterfactual Explanations](https://mlsanity.com/paper/1811.05245/) (2018)
- [Evaluating Robustness of Counterfactual Explanations](https://mlsanity.com/paper/2103.02354/) (2021)
- [CERTIFAI: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence models](https://mlsanity.com/paper/1905.07857/) (2019)
- [On the computation of counterfactual explanations -- A survey](https://mlsanity.com/paper/1911.07749/) (2019)
- [Conditional Generative Models for Counterfactual Explanations](https://mlsanity.com/paper/2101.10123/) (2021)
- [Learning Accurate and Interpretable Decision Rule Sets from Neural Networks](https://mlsanity.com/paper/2103.02826/) (2021)
- [Amortized Generation of Sequential Algorithmic Recourses for Black-box Models](https://mlsanity.com/paper/2106.03962/) (2021)
- [Features of Explainability: How users understand counterfactual and causal explanations for categorical and continuous features in XAI](https://mlsanity.com/paper/2204.10152/) (2022)
- [Model-agnostic and Scalable Counterfactual Explanations via Reinforcement Learning](https://mlsanity.com/paper/2106.02597/) (2021)
- [Model-Based Counterfactual Synthesizer for Interpretation](https://mlsanity.com/paper/2106.08971/) (2021)
- [Keep Your Friends Close and Your Counterfactuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Setting](https://mlsanity.com/paper/2205.05515/) (2022)
- [Let's Go to the Alien Zoo: Introducing an Experimental Framework to Study Usability of Counterfactual Explanations for Machine Learning](https://mlsanity.com/paper/2205.03398/) (2022)
- [Understanding Consumer Preferences for Explanations Generated by XAI Algorithms](https://mlsanity.com/paper/2107.02624/) (2021)

## Built on by (most cited first, 4 of 4)

- [Towards Unifying Evaluation of Counterfactual Explanations: Leveraging Large Language Models for Human-Centric Assessments](https://mlsanity.com/paper/2410.21131/) (2024)
- [When Bias Backfires: The Modulatory Role of Counterfactual Explanations on the Adoption of Algorithmic Bias in XAI-Supported Human Decision-Making](https://mlsanity.com/paper/2505.14377/) (2025)
- [Comparables XAI: Faithful Example-based AI Explanations with Counterfactual Trace Adjustments](https://mlsanity.com/paper/2602.13784/) (2026)
- [Integrating Feature Understandability into Feature Selection for Human-Centred Tabular Explanations](https://mlsanity.com/paper/2610.16627/) (2026)

## Abstract (arXiv)

Counterfactual explanations (CFEs) are a popular approach in explainable artificial intelligence (xAI), highlighting changes to input data necessary for altering a model's output. A CFE can either describe a scenario that is better than the factual state (upward CFE), or a scenario that is worse than the factual state (downward CFE). However, potential benefits and drawbacks of the directionality of CFEs for user behavior in xAI remain unclear. The current user study (N=161) compares the impact of CFE directionality on behavior and experience of participants tasked to extract new knowledge from an automated system based on model predictions and CFEs. Results suggest that upward CFEs provide a significant performance advantage over other forms of counterfactual feedback. Moreover, the study highlights potential benefits of mixed CFEs improving user performance compared to downward CFEs or no explanations. In line with the performance results, users' explicit knowledge of the system is statistically higher after receiving upward CFEs compared to downward comparisons. These findings imply that the alignment between explanation and task at hand, the so-called regulatory fit, may play a crucial role in determining the effectiveness of model explanations, informing future research directions in xAI. To ensure reproducible research, the entire code, underlying models and user data of this study is openly available: https://github.com/ukuhl/DirectionalAlienZoo