mlsanityData to 14 Oct 2026

Adversarial Counterfactual Visual Explanations

arXiv 2303.09962 ↗First posted 17 Mar 2023cs.CVcited by 47

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Counterfactual explanations and adversarial attacks have a related goal: flipping output labels with minimal perturbations regardless of their characteristics. Yet, adversarial attacks cannot be used directly in a counterfactual explanation perspective, as such perturbations are perceived as noise and not as actionable and understandable image modifications. Building on the robust learning literature, this paper proposes an elegant method to turn adversarial attacks into semantically meaningful perturbations, without modifying the classifiers to explain. The proposed approach hypothesizes that Denoising Diffusion Probabilistic Models are excellent regularizers for avoiding high-frequency and out-of-distribution perturbations when generating adversarial attacks. The paper's key idea is to build attacks through a diffusion model to polish them. This allows studying the target model regardless of its robustification level. Extensive experimentation shows the advantages of our counterfactual explanation approach over current State-of-the-Art in multiple testbeds.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 62 of its 75 references that are in the map. First posted on arXiv 17 Mar 2023.

Builds on

62 of 75 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Diffusion Models for Counterfactual Explanations29 Mar
  2. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  3. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 59

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  9. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  10. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  11. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  12. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
38 more
  1. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  2. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  3. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 939
  4. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  5. 2018Robustness May Be at Odds with Accuracycited by 880
  6. 2016Universal adversarial perturbationscited by 784
  7. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  8. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  9. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
  10. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 582
  11. 2022Diffusion Models for Adversarial Purificationcited by 476
  12. 2019Counterfactual Visual Explanationscited by 313
  13. 2019Interpretable Counterfactual Explanations Guided by Prototypescited by 223
  14. 2017Generating Natural Adversarial Examplescited by 183
  15. 2017Generative Adversarial Perturbationscited by 153
  16. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  17. 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
  18. 2020Explainable Object-induced Action Decision for Autonomous Vehiclescited by 88
  19. 2022B-cos Networks: Alignment is All We Need for Interpretabilitycited by 88
  20. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  21. 2019Explanation by Progressive Exaggerationcited by 76
  22. 2020Interpretable and Accurate Fine-grained Recognition via Region Groupingcited by 75
  23. 2021Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanationscited by 56
  24. 2021Convolutional Dynamic Alignment Networks for Interpretable Classificationscited by 54
  25. 2020SCOUT: Self-aware Discriminant Counterfactual Explanationscited by 52
  26. 2021Exploring Counterfactual Explanations Through the Lens of Adversarial Examples: A Theoretical and Empirical Analysiscited by 52
  27. 2020GANterfactual - Counterfactual Explanations for Medical Non-Experts using Generative Adversarial Learningcited by 50
  28. 2021Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertaintiescited by 41
  29. 2021STEEX: Steering Counterfactual Explanations with Semanticscited by 40
  30. 2021A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Conceptscited by 33
  31. 2022Cycle-Consistent Counterfactuals by Latent Transformationscited by 33
  32. 2022Making Heads or Tails: Towards Semantically Consistent Visual Counterfactualscited by 30
  33. 2022Sparse Visual Counterfactual Explanations in Image Spacecited by 30
  34. 2020CounteRGAN: Generating Realistic Counterfactuals with Residual Generative Adversarial Netscited by 29
  35. 2021Conditional Generative Models for Counterfactual Explanationscited by 28
  36. 2021CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliencycited by 28
  37. 2021Enhancing Adversarial Robustness via Test-time Transformation Ensemblingcited by 27
  38. 2021ECINN: Efficient Counterfactuals from Invertible Neural Networkscited by 26

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 12; 2025: 15; 2026: 15.

47 papers.

Sort:

By area:

  1. 2023LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Imagescited by 46
  2. 2023Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generationcited by 41
  3. 2023Text-to-Image Models for Counterfactual Explanations: a Black-Box Approachcited by 28
  4. 2023Latent Diffusion Counterfactual Explanationscited by 17
  5. 2024Global Counterfactual Directionscited by 15
  6. 2024Rethinking Visual Counterfactual Explanations Through Region Constraintcited by 14
  7. 2024Diffusion-based Iterative Counterfactual Explanations for Fetal Ultrasound Image Quality Assessmentcited by 13
  8. 2024Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discoverycited by 8
  9. 2024Contrast Sets for Evaluating Language-Guided Robot Policiescited by 7
  10. 2025Towards Desiderata-Driven Design of Visual Counterfactual Explainerscited by 7
  11. 2025Mitigating Clever Hans Strategies in Image Classifiers through Generating Counterexamplescited by 7
  12. 2025Causally Steered Diffusion for Automated Video Counterfactual Generationcited by 6
35 more
  1. 2023Designing User-Centric Behavioral Interventions to Prevent Dysglycemia with Novel Counterfactual Explanationscited by 5
  2. 2025Diffusion Counterfactuals for Image Regressorscited by 5
  3. 2025BlurGuard: A Simple Approach for Robustifying Image Protection Against AI-Powered Editingcited by 5
  4. 2024CoLa-DCE -- Concept-guided Latent Diffusion Counterfactual Explanationscited by 4
  5. 2024Unsupervised Model Diagnosiscited by 4
  6. 2025From Visual Explanations to Counterfactual Explanations with Latent Diffusioncited by 4
  7. 2025V-CECE: Visual Counterfactual Explanations via Conceptual Editscited by 4
  8. 2026Visual Disentangled Diffusion Autoencoders: Scalable Counterfactual Generation for Foundation Modelscited by 4
  9. 2025Imbalanced Classification through the Lens of Spurious Correlationscited by 3
  10. 2025Learning Common and Salient Generative Factors Between Two Image Datasetscited by 3
  11. 2026Protein Counterfactuals via Diffusion-Guided Latent Optimizationcited by 3
  12. 2026SCE-LITE-HQ: Smooth visual counterfactual explanations with generative foundation modelscited by 3
  13. 2024COIN: Counterfactual inpainting for weakly supervised semantic segmentation for medical imagescited by 2
  14. 2024Derivative-Free Diffusion Manifold-Constrained Gradient for Unified XAIcited by 2
  15. 2024GIFT: A Framework Towards Global Interpretable Faithful Textual Explanations of Vision Classifierscited by 2
  16. 2024OMENN: One Matrix to Explain Neural Networkscited by 2
  17. 2026MaskDiME: Adaptive Masked Diffusion for Precise and Efficient Visual Counterfactual Explanationscited by 2
  18. 2026Vision-language models for chest radiography do not always need the imagecited by 2
  19. 2024Explaining an image classifier with a generative model conditioned by uncertaintycited by 1
  20. 2025Disentangling Visual Transformers: Patch-level Interpretability for Image Classificationcited by 1
  21. 2025Counterfactual Visual Explanation via Causally-Guided Adversarial Steeringcited by 1
  22. 2025Back to the Feature: Explaining Video Classifiers with Video Counterfactual Explanationscited by 1
  23. 2026Faithful, Sufficient and Understandable: Rethinking Graph Counterfactual Explanations via Discrete Diffusion Inversioncited by 1
  24. 2026When the Edit Changes the Patient: Measuring Identity Preservation in Counterfactual Retinal Imagescited by 1
  25. 2025Discovering Concept Directions from Diffusion-based Counterfactuals via Latent Clusteringcited by 0
  26. 2025Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering Human Perceptual Variability on Facial Expressionscited by 0
  27. 2025Cross-modal Counterfactual Explanations: Uncovering Decision Factors and Dataset Biases in Subjective Classificationcited by 0
  28. 2026U-CECE: A Universal Multi-Resolution Framework for Conceptual Counterfactual Explanationscited by 0
  29. 2026Clinically-Grounded Counterfactual Reasoning for Medical Video Diagnosiscited by 0
  30. 2026Concept-based Visual Counterfactual Explanations with Diffusion Modelscited by 0
  31. 2026FiRe: Fixed-Noise Refinement for Visual Counterfactual Explanationscited by 0
  32. 2026Counterfactual Contrastive Analysiscited by 0
  33. 2026From Interpretability Methods to Interpretable Modelscited by 0
  34. 2026MedDiME: Efficient Latent Diffusion with Adaptive Masking for Medical Counterfactual Generationcited by 0
  35. 2026Towards Fast and Disentangled Counterfactuals for Visual Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Do Counterfactual Examples Complicate Adversarial Training?
  2. 2024Adversarial Examples are Misaligned in Diffusion Model Manifolds
  3. 2023Diffusion Models for Imperceptible and Transferable Adversarial Attack
  4. 2023Semantic Adversarial Attacks via Diffusion Models
  5. 2023Diffusion-based Visual Counterfactual Explanations -- Towards Systematic Quantitative Evaluation
  6. 2025Beyond Classification: Evaluating Diffusion Denoised Smoothing for Security-Utility Trade off
  7. 2024CausAdv: A Causal-based Framework for Detecting Adversarial Examples
  8. 2024AdvLogo: Adversarial Patch Attack against Object Detectors based on Diffusion Models