mlsanityData to 14 Oct 2026

The Manifold Hypothesis for Gradient-Based Explanations

arXiv 2206.07387 ↗First posted 15 Jun 2022cs.LG cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

When do gradient-based explanation algorithms provide perceptually-aligned explanations? We propose a criterion: the feature attributions need to be aligned with the tangent space of the data manifold. To provide evidence for this hypothesis, we introduce a framework based on variational autoencoders that allows to estimate and generate image manifolds. Through experiments across a range of different datasets -- MNIST, EMNIST, CIFAR10, X-ray pneumonia and Diabetic Retinopathy detection -- we demonstrate that the more a feature attribution is aligned with the tangent space of the data, the more perceptually-aligned it tends to be. We then show that the attributions provided by popular post-hoc methods such as Integrated Gradients and SmoothGrad are more strongly aligned with the data manifold than the raw gradient. Adversarial training also improves the alignment of model gradients with the data manifold. As a consequence, we suggest that explanation algorithms should actively strive to align their explanations with the data manifold. This is an extended version of a CVPR Workshop paper. Code is available at https://github.com/tml-tuebingen/explanations-manifold.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 47 of its 73 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

47 of 73 references.

References · 47

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  8. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  9. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  10. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  11. 2018Sanity Checks for Saliency Mapscited by 1,156
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,109
35 more
  1. 2020Concept Bottleneck Modelscited by 1,032
  2. 2014Striving for Simplicity: The All Convolutional Netcited by 990
  3. 2018Robustness May Be at Odds with Accuracycited by 880
  4. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  5. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
  6. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
  7. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 426
  8. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
  9. 2018Understanding disentangling in β-VAEcited by 335
  10. 2017The (Un)reliability of saliency methodscited by 333
  11. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 268
  12. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  13. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  14. 2019Explanations can be manipulated and geometry is to blamecited by 216
  15. 2018Disentangling Adversarial Robustness and Generalizationcited by 172
  16. 2020Debugging Tests for Model Explanationscited by 147
  17. 2018Explaining Image Classifiers by Counterfactual Generationcited by 144
  18. 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112
  19. 2019CXPlain: Causal Explanations for Model Interpretation under Uncertaintycited by 98
  20. 2020Shapley explainability on the data manifoldcited by 88
  21. 2017The Riemannian Geometry of Deep Generative Modelscited by 82
  22. 2019Explanation by Progressive Exaggerationcited by 76
  23. 2018How Important Is a Neuron?cited by 73
  24. 2018How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanationcited by 72
  25. 2020Fairwashing Explanations with Off-Manifold Detergentcited by 72
  26. 2021Do Input Gradients Highlight Discriminative Features?cited by 52
  27. 2021Exploring Counterfactual Explanations Through the Lens of Adversarial Examples: A Theoretical and Empirical Analysiscited by 52
  28. 2021Towards the Unification and Robustness of Perturbation and Gradient Based Explanationscited by 47
  29. 2019Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?cited by 43
  30. 2020Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imagingcited by 32
  31. 2018Influence-Directed Explanations for Deep Convolutional Networkscited by 31
  32. 2016Lets keep it simple, Using simple architectures to outperform deeper and more complex architecturescited by 25
  33. 2019Bridging Adversarial Robustness and Gradient Interpretabilitycited by 22
  34. 2021What does LIME really see in images?cited by 21
  35. 2021Building Reliable Explanations of Unreliable Neural Networks: Locally Smoothing Perspective of Model Interpretationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 8; 2026: 2.

16 papers.

Sort:

By area:

  1. 2023Manifold Preserving Guided Diffusioncited by 157
  2. 2024Manifold Integrated Gradients: Riemannian Geometry for Feature Attributioncited by 21
  3. 2023Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustnesscited by 20
  4. 2024Explainable Molecular Property Prediction: Aligning Chemical Concepts with Predictions via Language Modelscited by 3
  5. 2024Geometric Remove-and-Retrain (GOAR): Coordinate-Invariant eXplainable AI Assessmentcited by 3
  6. 2025Tangentially Aligned Integrated Gradients for User-Friendly Explanationscited by 3
  7. 2025EquiReg: Equivariance Regularized Diffusion for Inverse Problemscited by 3
  8. 2025Riemannian Integrated Gradients: A Geometric View of Explainable AIcited by 2
  9. 2025Measuring the (Un)Faithfulness of Concept-Based Explanationscited by 2
  10. 2025Latent Guidance in Diffusion Models for Perceptual Evaluationscited by 2
  11. 2026Manifold-Aligned Guided Integrated Gradients for Reliable Feature Attributioncited by 2
  12. 2025Generative Model Inversion Through the Lens of the Manifold Hypothesiscited by 1
4 more
  1. 2024Post-hoc and manifold explanations analysis of facial expression data based on deep learningcited by 0
  2. 2025Algebraic Adversarial Attacks on Explainability Modelscited by 0
  3. 2025Graph-based Integrated Gradients for Explaining Graph Neural Networkscited by 0
  4. 2026Concept-based explanation of gene expression prediction from H&E imagescited by 0