mlsanityData to 14 Oct 2026

Diffusion Visual Counterfactual Explanations

arXiv 2210.11841 ↗First posted 21 Oct 2022cs.CV cs.LGcited by 96

Not on the map. Why?

Cited by 24 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visual Counterfactual Explanations (VCEs) are an important tool to understand the decisions of an image classifier. They are 'small' but 'realistic' semantic changes of the image changing the classifier decision. Current approaches for the generation of VCEs are restricted to adversarially robust models and often contain non-realistic artefacts, or are limited to image classification problems with few classes. In this paper, we overcome this by generating Diffusion Visual Counterfactual Explanations (DVCEs) for arbitrary ImageNet classifiers via a diffusion process. Two modifications to the diffusion process are key for our DVCEs: first, an adaptive parameterization, whose hyperparameters generalize across images and models, together with distance regularization and late start of the diffusion process, allow us to generate images with minimal semantic changes to the original ones but different classification. Second, our cone regularization via an adversarially robust model ensures that the diffusion process does not converge to trivial non-semantic changes, but instead produces realistic images of the target class which achieve high confidence by the classifier.

Key facts. Cited by 96 papers in the map (data through 14 Oct 2026). Builds on 46 of its 67 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

46 of 67 references.

References · 46

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  11. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  12. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
34 more
  1. 2022Classifier-Free Diffusion Guidancecited by 5,847
  2. 2013Intriguing properties of neural networkscited by 5,155
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  7. 2022A ConvNet for the 2020scited by 3,579
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  9. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  10. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  11. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  12. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
  13. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  14. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
  15. 2022Understanding Diffusion Models: A Unified Perspectivecited by 340
  16. 2019Counterfactual Visual Explanationscited by 313
  17. 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 262
  18. 2019Interpretable Counterfactual Explanations Guided by Prototypescited by 223
  19. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 212
  20. 2020Deep Structural Causal Models for Tractable Counterfactual Inferencecited by 211
  21. 2019Full-Gradient Representation for Neural Network Visualizationcited by 191
  22. 2021Are Transformers More Robust Than CNNs?cited by 187
  23. 2016Generating Visual Explanationscited by 158
  24. 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
  25. 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
  26. 2019The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasonscited by 115
  27. 2018Grounding Visual Explanationscited by 93
  28. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  29. 2020Adversarial Robustness on In- and Out-Distribution Improves Explainabilitycited by 73
  30. 2021Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertaintiescited by 41
  31. 2020Smoothed Geometry for Robust Attributioncited by 38
  32. 2021Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Imagescited by 35
  33. 2022Cycle-Consistent Counterfactuals by Latent Transformationscited by 33
  34. 2022Sparse Visual Counterfactual Explanations in Image Spacecited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 18; 2024: 29; 2025: 30; 2026: 17.

96 papers.

Sort:

By area:

  1. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
  2. 2022Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNetcited by 41
  3. 2023Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generationcited by 41
  4. 2023Efficient and Scalable Graph Generation through Iterative Local Expansioncited by 36
  5. 2022OCTET: Object-aware Counterfactual Explanationscited by 28
  6. 2023Text-to-Image Models for Counterfactual Explanations: a Black-Box Approachcited by 28
  7. 2023Learning Data Representations with Joint Diffusion Modelscited by 22
  8. 2023From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modelingcited by 22
  9. 2024Counterfactual Image Editingcited by 20
  10. 2024Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approachcited by 18
  11. 2024Causal Diffusion Autoencoders: Toward Counterfactual Generation via Diffusion Probabilistic Modelscited by 18
  12. 2023Latent Diffusion Counterfactual Explanationscited by 17
38 more
  1. 2024Global Counterfactual Directionscited by 15
  2. 2023DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRIcited by 14
  3. 2023SDDM: Score-Decomposed Diffusion Models on Manifolds for Unpaired Image-to-Image Translationcited by 14
  4. 2024Robust image representations with counterfactual contrastive learningcited by 14
  5. 2024Rethinking Visual Counterfactual Explanations Through Region Constraintcited by 14
  6. 2023Natural Example-Based Explainability: a Surveycited by 13
  7. 2023DiG-IN: Diffusion Guidance for Investigating Networks -- Uncovering Classifier Differences Neuron Visualisations and Visual Counterfactual Explanationscited by 11
  8. 2024Counterfactual contrastive learning: robust representations via causal image synthesiscited by 11
  9. 2024Probabilistically Plausible Counterfactual Explanations with Normalizing Flowscited by 11
  10. 2024Unpaired Photo-realistic Image Deraining with Energy-informed Diffusion Modelcited by 11
  11. 2023Emerging Synergies in Causality and Deep Generative Models: A Surveycited by 10
  12. 2025Diffusion Counterfactual Generation with Semantic Abductioncited by 10
  13. 2023Discriminative Class Tokens for Text-to-Image Diffusion Modelscited by 9
  14. 2024A survey on Concept-based Approaches For Model Improvementcited by 8
  15. 2024Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discoverycited by 8
  16. 2024SAFE-RL: Saliency-Aware Counterfactual Explainer for Deep Reinforcement Learning Policiescited by 8
  17. 2024Benchmarking XAI Explanations with Human-Aligned Evaluationscited by 8
  18. 2024Interval Abstractions for Robust Counterfactual Explanationscited by 7
  19. 2024GradCheck: Analyzing classifier guidance gradients for conditional diffusion samplingcited by 7
  20. 2025Towards Desiderata-Driven Design of Visual Counterfactual Explainerscited by 7
  21. 2025Mitigating Clever Hans Strategies in Image Classifiers through Generating Counterexamplescited by 7
  22. 2023Diffusion-based Visual Counterfactual Explanations -- Towards Systematic Quantitative Evaluationcited by 6
  23. 2024Benchmarking Segmentation Models with Mask-Preserved Attribute Editingcited by 6
  24. 2025Causally Steered Diffusion for Automated Video Counterfactual Generationcited by 6
  25. 2023SAFE: Saliency-Aware Counterfactual Explanations for DNN-based Automated Driving Systemscited by 5
  26. 2023Navigating the Structured What-If Spaces: Counterfactual Generation via Structured Diffusioncited by 5
  27. 2024Feature Accentuation: Revealing 'What' Features Respond to in Natural Imagescited by 5
  28. 2024GUIDE: Guidance-based Incremental Learning with Diffusion Modelscited by 5
  29. 2024What could go wrong? Discovering and describing failure modes in computer visioncited by 5
  30. 2025Diffusion Models for Computational Neuroimaging: A Surveycited by 5
  31. 2024CoLa-DCE -- Concept-guided Latent Diffusion Counterfactual Explanationscited by 4
  32. 2024Unsupervised Model Diagnosiscited by 4
  33. 2025From Visual Explanations to Counterfactual Explanations with Latent Diffusioncited by 4
  34. 2025Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifierscited by 4
  35. 2025V-CECE: Visual Counterfactual Explanations via Conceptual Editscited by 4
  36. 2026Visual Disentangled Diffusion Autoencoders: Scalable Counterfactual Generation for Foundation Modelscited by 4
  37. 2023Generating Realistic Counterfactuals for Retinal Fundus and OCT Images using Diffusion Modelscited by 3
  38. 2026Protein Counterfactuals via Diffusion-Guided Latent Optimizationcited by 3

50 most cited of 96 All 96 (JSON)