# Why Does My Model Fail? Contrastive Local Explanations for Retail Forecasting

arXiv: https://arxiv.org/abs/1908.00085 · HTML: https://mlsanity.com/paper/1908.00085/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2019.

## Built on by (most cited first, 30 of 31)

- [Benchmarking and Survey of Explanation Methods for Black Box Models](https://mlsanity.com/paper/2102.13076/) (2021)
- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021)
- [CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networks](https://mlsanity.com/paper/2102.03322/) (2021)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniques](https://mlsanity.com/paper/2103.01035/) (2021)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications](https://mlsanity.com/paper/2103.04244/) (2021)
- [Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI](https://mlsanity.com/paper/2206.10847/) (2022)
- [A Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation, and Research Challenges](https://mlsanity.com/paper/2210.12089/) (2022)
- [Selective Explanations: Leveraging Human Input to Align Explainable AI](https://mlsanity.com/paper/2301.09656/) (2023)
- [How Well do Feature Visualizations Support Causal Understanding of CNN Activations?](https://mlsanity.com/paper/2106.12447/) (2021)
- [Machine Explanations and Human Understanding](https://mlsanity.com/paper/2202.04092/) (2022)
- [Uncertainty Estimation and Out-of-Distribution Detection for Counterfactual Explanations: Pitfalls and Solutions](https://mlsanity.com/paper/2107.09734/) (2021)
- [Features of Explainability: How users understand counterfactual and causal explanations for categorical and continuous features in XAI](https://mlsanity.com/paper/2204.10152/) (2022)
- [Counterfactual Explanations for Misclassified Images: How Human and Machine Explanations Differ](https://mlsanity.com/paper/2212.08733/) (2022)
- [Explaining Groups of Instances Counterfactually for XAI: A Use Case, Algorithm and User Study for Group-Counterfactuals](https://mlsanity.com/paper/2303.09297/) (2023)
- [Decoding AI's Nudge: A Unified Framework to Predict Human Behavior in AI-assisted Decision Making](https://mlsanity.com/paper/2401.05840/) (2024)
- [Incremental XAI: Memorable Understanding of AI with Incremental Explanations](https://mlsanity.com/paper/2404.06733/) (2024)
- [Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box](https://mlsanity.com/paper/2203.02399/) (2022)
- [Anomaly Attribution with Likelihood Compensation](https://mlsanity.com/paper/2208.10679/) (2022)
- [To Trust or Not to Trust a Regressor: Estimating and Explaining Trustworthiness of Regression Predictions](https://mlsanity.com/paper/2104.06982/) (2021)
- [Justifying Social-Choice Mechanism Outcome for Improving Participant Satisfaction](https://mlsanity.com/paper/2205.15863/) (2022)
- [Counterfactual Explainer Framework for Deep Reinforcement Learning Models Using Policy Distillation](https://mlsanity.com/paper/2305.16532/) (2023)
- [Causal Explanations for Sequential Decision Making Under Uncertainty](https://mlsanity.com/paper/2205.15462/) (2022)
- [OpenHEXAI: An Open-Source Framework for Human-Centered Evaluation of Explainable Machine Learning](https://mlsanity.com/paper/2403.05565/) (2024)
- [Good Data Is All Imitation Learning Needs](https://mlsanity.com/paper/2409.17605/) (2024)
- [Looking in the mirror: A faithful counterfactual explanation method for interpreting deep image classification models](https://mlsanity.com/paper/2509.16822/) (2025)
- [Can counterfactual explanations of AI systems' predictions skew lay users' causal intuitions about the world? If so, can we correct for that?](https://mlsanity.com/paper/2205.06241/) (2022)
- [XpertAI: uncovering regression model strategies for sub-manifolds](https://mlsanity.com/paper/2403.07486/) (2024)

## Abstract (arXiv)

In various business settings, there is an interest in using more complex machine learning techniques for sales forecasting. It is difficult to convince analysts, along with their superiors, to adopt these techniques since the models are considered to be "black boxes," even if they perform better than current models in use. We examine the impact of contrastive explanations about large errors on users' attitudes towards a "black-box'" model. We propose an algorithm, Monte Carlo Bounds for Reasonable Predictions. Given a large error, MC-BRP determines (1) feature values that would result in a reasonable prediction, and (2) general trends between each feature and the target, both based on Monte Carlo simulations. We evaluate on a real dataset with real users by conducting a user study with 75 participants to determine if explanations generated by MC-BRP help users understand why a prediction results in a large error, and if this promotes trust in an automatically-learned model. Our study shows that users are able to answer objective questions about the model's predictions with overall 81.1% accuracy when provided with these contrastive explanations. We show that users who saw MC-BRP explanations understand why the model makes large errors in predictions significantly more than users in the control group. We also conduct an in-depth analysis on the difference in attitudes between Practitioners and Researchers, and confirm that our results hold when conditioning on the users' background.