mlsanityData to 7 Oct 2026

Counterfactual Explanations for Misclassified Images: How Human and Machine Explanations Differ

arXiv 2212.08733 ↗First posted 16 Dec 2022cs.LG cs.AI cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Counterfactual explanations have emerged as a popular solution for the eXplainable AI (XAI) problem of elucidating the predictions of black-box deep-learning systems due to their psychological validity, flexibility across problem domains and proposed legal compliance. While over 100 counterfactual methods exist, claiming to generate plausible explanations akin to those preferred by people, few have actually been tested on users (~7%). So, the psychological validity of these counterfactual algorithms for effective XAI for image data is not established. This issue is addressed here using a novel methodology that (i) gathers ground truth human-generated counterfactual explanations for misclassified images, in two user studies and, then, (ii) compares these human-generated ground-truth explanations to computationally-generated explanations for the same misclassifications. Results indicate that humans do not "minimally edit" images when generating counterfactual explanations. Instead, they make larger, "meaningful" edits that better approximate prototypes in the counterfactual class.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 62 of its 122 references that are in the map. First posted on arXiv 16 Dec 2022.

Builds on

62 of 122 references.

References · 62

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  8. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  9. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  10. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  11. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  12. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
38 more
  1. 2018Sanity Checks for Saliency Mapscited by 1,149
  2. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  3. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  4. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  5. 2016The Mythos of Model Interpretabilitycited by 716
  6. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 612
  7. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  8. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 343
  9. 2019Counterfactual Visual Explanationscited by 312
  10. 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 261
  11. 2018Metrics for Explainable AI: Challenges and Prospectscited by 257
  12. 2020Algorithmic Recourse: from Counterfactual Explanations to Interventionscited by 250
  13. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  14. 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
  15. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  16. 2019Interpretable Counterfactual Explanations Guided by Prototypescited by 223
  17. 2019The What-If Tool: Interactive Probing of Machine Learning Modelscited by 207
  18. 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
  19. 2020Algorithmic recourse under imperfect causal knowledge: a probabilistic approachcited by 139
  20. 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 138
  21. 2019Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systemscited by 128
  22. 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
  23. 2020Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectorscited by 119
  24. 2019The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasonscited by 115
  25. 2019Efficient Search for Diverse Coherent Explanationscited by 111
  26. 2019The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanationscited by 97
  27. 2021If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniquescited by 82
  28. 2020Good Counterfactuals and Where to Find Them: A Case-Based Technique for Generating Counterfactuals for Explainable AI (XAI)cited by 79
  29. 2021Input Similarity from the Neural Network Perspectivecited by 78
  30. 2015The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classificationcited by 76
  31. 2019Explanation by Progressive Exaggerationcited by 76
  32. 2021IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammographycited by 72
  33. 2019An Evaluation of the Human-Interpretability of Explanationcited by 71
  34. 2020Instance-based Counterfactual Explanations for Time Series Classificationcited by 64
  35. 2019Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AIcited by 63
  36. 2020On Generating Plausible Counterfactual and Semi-Factual Explanations for Deep Learningcited by 60
  37. 2018Interpretable Credit Application Predictions With Counterfactual Explanationscited by 43
  38. 2021Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertaintiescited by 41

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 5.

12 papers.

  1. 2025Can LLMs Explain Themselves Counterfactually?cited by 10
  2. 2024Sparse and Faithful Explanations Without Sparse Modelscited by 5
  3. 2024"What is Different Between These Datasets?" A Framework for Explaining Data Distribution Shiftscited by 5
  4. 2026What-If Explanations Over Time: Counterfactuals for Time Series Classificationcited by 4
  5. 2026Comparables XAI: Faithful Example-based AI Explanations with Counterfactual Trace Adjustmentscited by 1
  6. 2024Exploring the Effect of Explanation Content and Format on User Comprehension and Trust in Healthcarecited by 0
  7. 2024Improving Decision Sparsitycited by 0
  8. 2025Feature-Guided Neighbor Selection for Non-Expert Evaluation of Model Predictionscited by 0
  9. 2025An Information-Flow Perspective on Explainability Requirements: Specification and Verificationcited by 0
  10. 2026Improving understanding and trust in AI: How users benefit from interval-based counterfactual explanationscited by 0
  11. 2026Addressing the Selection Problem in Explainable AIcited by 0
  12. 2026Diversified and Perceptible Counterfactual Examples Leveraging Expert Knowledgecited by 0