mlsanityData to 7 Oct 2026

Sparse Visual Counterfactual Explanations in Image Space

arXiv 2205.07972 ↗First posted 16 May 2022cs.CVcited by 30

Not on the map. Why?

Abstract (arXiv)

Visual counterfactual explanations (VCEs) in image space are an important tool to understand decisions of image classifiers as they show under which changes of the image the decision of the classifier would change. Their generation in image space is challenging and requires robust models due to the problem of adversarial examples. Existing techniques to generate VCEs in image space suffer from spurious changes in the background. Our novel perturbation model for VCEs together with its efficient optimization via our novel Auto-Frank-Wolfe scheme yields sparse VCEs which lead to subtle changes specific for the target class. Moreover, we show that VCEs can be used to detect undesired behavior of ImageNet classifiers due to spurious features in the ImageNet dataset.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 53 of its 72 references that are in the map. First posted on arXiv 16 May 2022.

Builds on

53 of 72 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2017On Calibration of Modern Neural Networkscited by 4,860
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  12. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
38 more
  1. 2019Natural Adversarial Examplescited by 1,401
  2. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  3. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  4. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  5. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  6. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  7. 2018Robustness May Be at Odds with Accuracycited by 873
  8. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  9. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  10. 2018Recognition in Terra Incognitacited by 655
  11. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 612
  12. 2017Interpretation of Neural Networks is Fragilecited by 443
  13. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
  14. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  15. 2019Counterfactual Visual Explanationscited by 312
  16. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  17. 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 261
  18. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  19. 2019Explanations can be manipulated and geometry is to blamecited by 215
  20. 2020Deep Structural Causal Models for Tractable Counterfactual Inferencecited by 210
  21. 2018Do CIFAR-10 Classifiers Generalize to CIFAR-10?cited by 197
  22. 2019Full-Gradient Representation for Neural Network Visualizationcited by 189
  23. 2016Generating Visual Explanationscited by 156
  24. 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
  25. 2020Perceptual Adversarial Robustness: Defense Against Unseen Threat Modelscited by 137
  26. 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
  27. 2019The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasonscited by 115
  28. 2021Large-scale Unsupervised Semantic Segmentationcited by 98
  29. 2018Grounding Visual Explanationscited by 92
  30. 2022Diffusion Causal Models for Counterfactual Estimationcited by 87
  31. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  32. 2020Adversarial Robustness on In- and Out-Distribution Improves Explainabilitycited by 73
  33. 2019Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizationscited by 72
  34. 2020Understanding Failures of Deep Networks via Robust Feature Extractioncited by 56
  35. 2020Interpretability and Explainability: A Machine Learning Zoo Mini-tourcited by 55
  36. 2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 50
  37. 2021Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertaintiescited by 41
  38. 2020Smoothed Geometry for Robust Attributioncited by 38

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 9; 2024: 7; 2025: 6; 2026: 5.

30 papers.

Sort:

By area:

  1. 2022Diffusion Visual Counterfactual Explanationscited by 96
  2. 2023Adversarial Counterfactual Visual Explanationscited by 47
  3. 2022Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNetcited by 41
  4. 2023Latent Diffusion Counterfactual Explanationscited by 17
  5. 2024Global Counterfactual Directionscited by 15
  6. 2024Rethinking Visual Counterfactual Explanations Through Region Constraintcited by 14
  7. 2023DiG-IN: Diffusion Guidance for Investigating Networks -- Uncovering Classifier Differences Neuron Visualisations and Visual Counterfactual Explanationscited by 11
  8. 2023Emerging Synergies in Causality and Deep Generative Models: A Surveycited by 10
  9. 2025Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metricscited by 10
  10. 2023Discriminative Class Tokens for Text-to-Image Diffusion Modelscited by 9
  11. 2023Deep Backtracking Counterfactuals for Causally Compliant Explanationscited by 8
  12. 2023Diffusion-based Visual Counterfactual Explanations -- Towards Systematic Quantitative Evaluationcited by 6
18 more
  1. 2023Navigating the Structured What-If Spaces: Counterfactual Generation via Structured Diffusioncited by 5
  2. 2024Sparse and Faithful Explanations Without Sparse Modelscited by 5
  3. 2024Feature Accentuation: Revealing 'What' Features Respond to in Natural Imagescited by 5
  4. 2024CoLa-DCE -- Concept-guided Latent Diffusion Counterfactual Explanationscited by 4
  5. 2025From Visual Explanations to Counterfactual Explanations with Latent Diffusioncited by 4
  6. 2023Generating Realistic Counterfactuals for Retinal Fundus and OCT Images using Diffusion Modelscited by 3
  7. 2025Robustness in Both Domains: CLIP Needs a Robust Text Encodercited by 2
  8. 2024Do Counterfactual Examples Complicate Adversarial Training?cited by 1
  9. 2025Scalable Energy-Based Models via Adversarial Training: Unifying Discrimination and Generationcited by 1
  10. 2025Back to the Feature: Explaining Video Classifiers with Video Counterfactual Explanationscited by 1
  11. 2026Which Modality Decides? Counterfactual Modality Attribution for Multimodal LLMscited by 1
  12. 2026When the Edit Changes the Patient: Measuring Identity Preservation in Counterfactual Retinal Imagescited by 1
  13. 2022Clarity: an improved gradient method for producing quality visual counterfactual explanationscited by 0
  14. 2024Improving Decision Sparsitycited by 0
  15. 2025SafeFix: Targeted Model Repair via Controlled Image Generationcited by 0
  16. 2026Exploring Sparsity and Smoothness of Arbitrary Lp Norms in Adversarial Attackscited by 0
  17. 2026FiRe: Fixed-Noise Refinement for Visual Counterfactual Explanationscited by 0
  18. 2026Targeted Visual Counterfactual Explanations for Contrastive Vision-Language Modelcited by 0