mlsanityData to 7 Oct 2026

Counterfactual Visual Explanations

arXiv 1904.07451 ↗First posted 16 Apr 2019cs.LG cs.AI cs.CVcited by 312

Not on the map. Why?

Abstract (arXiv)

In this work, we develop a technique to produce counterfactual visual explanations. Given a 'query' image I for which a vision system predicts class c, a counterfactual visual explanation identifies how I could change such that the system would output a different specified class c'. To do this, we select a 'distractor' image I' that the system predicts as class c' and identify spatial regions in I and I' such that replacing the identified region in I with the identified region in I' would push the system towards classifying I as c'. We apply our approach to multiple image classification datasets generating qualitative results showcasing the interpretability and discriminativeness of our counterfactual explanations. To explore the effectiveness of our explanations in teaching humans, we present machine teaching experiments for the task of fine-grained bird classification. We find that users trained to distinguish bird species fare better when given access to counterfactual explanations in addition to training examples.

Key facts. Cited by 312 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 71; 2022: 61; 2023: 74; 2024: 48; 2025: 37; 2026: 21.

312 papers.

Sort:

By area:

  1. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  2. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  3. 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
  4. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  5. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  6. 2023Probabilistic Concept Bottleneck Modelscited by 101
  7. 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
  8. 2022Diffusion Visual Counterfactual Explanationscited by 96
  9. 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 87
  10. 2022Diffusion Causal Models for Counterfactual Estimationcited by 87
  11. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  12. 2021If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniquescited by 82
38 more
  1. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
  2. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  3. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  4. 2022Diffusion Models for Counterfactual Explanationscited by 79
  5. 2022Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoningcited by 78
  6. 2023Fast Model Debias with Machine Unlearningcited by 70
  7. 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
  8. 2021Editing a classifier by rewriting its prediction rulescited by 64
  9. 2022"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interactioncited by 63
  10. 2021Counterfactual Explainable Recommendationcited by 60
  11. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  12. 2025CF-VLM:CounterFactual Vision-Language Fine-tuningcited by 57
  13. 2021Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanationscited by 56
  14. 2021Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanationscited by 53
  15. 2022Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigationcited by 50
  16. 2023FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methodscited by 49
  17. 2023Adversarial Counterfactual Visual Explanationscited by 47
  18. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  19. 2021DRIVE: Deep Reinforced Accident Anticipation with Visual Explanationcited by 46
  20. 2022Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capabilitycited by 46
  21. 2023LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Imagescited by 46
  22. 2021STEEX: Steering Counterfactual Explanations with Semanticscited by 40
  23. 2021Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learningcited by 38
  24. 2021Finding and Fixing Spurious Patterns with Explanationscited by 38
  25. 2023Learning Support and Trivial Prototypes for Interpretable Image Classificationcited by 38
  26. 2021DISSECT: Disentangled Simultaneous Explanations via Concept Traversalscited by 37
  27. 2024Vinoground: Scrutinizing LMMs over Dense Temporal Reasoning with Short Videoscited by 37
  28. 2022Missingness Bias in Model Debuggingcited by 36
  29. 2022First is Better Than Last for Language Data Influencecited by 34
  30. 2022Cycle-Consistent Counterfactuals by Latent Transformationscited by 33
  31. 2023ProtoSeg: Interpretable Semantic Segmentation with Prototypical Partscited by 33
  32. 2021HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizationcited by 31
  33. 20213DB: A Framework for Debugging Computer Vision Modelscited by 31
  34. 2021Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysiscited by 30
  35. 2022Making Heads or Tails: Towards Semantically Consistent Visual Counterfactualscited by 30
  36. 2022Sparse Visual Counterfactual Explanations in Image Spacecited by 30
  37. 2023GaitGCI: Generative Counterfactual Intervention for Gait Recognitioncited by 30
  38. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30

50 most cited of 312 All 312 (JSON)