mlsanityData to 7 Oct 2026

A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations

arXiv 1805.07039 ↗First posted 18 May 2018cs.CV cs.AIcited by 59

Not on the map. Why?

Abstract (arXiv)

Backpropagation-based visualizations have been proposed to interpret convolutional neural networks (CNNs), however a theory is missing to justify their behaviors: Guided backpropagation (GBP) and deconvolutional network (DeconvNet) generate more human-interpretable but less class-sensitive visualizations than saliency map. Motivated by this, we develop a theoretical explanation revealing that GBP and DeconvNet are essentially doing (partial) image recovery which is unrelated to the network decisions. Specifically, our analysis shows that the backward ReLU introduced by GBP and DeconvNet, and the local connections in CNNs are the two main causes of compelling visualizations. Extensive experiments are provided that support the theoretical analysis.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 19; 2023: 10; 2024: 7; 2025: 5; 2026: 3.

59 papers.

Sort:

  1. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  2. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  3. 2022Impossibility Theorems for Feature Attributioncited by 78
  4. 2022Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanationscited by 77
  5. 2022Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlationcited by 59
  6. 2022Framework for Evaluating Faithfulness of Local Explanationscited by 55
  7. 2021Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networkscited by 48
  8. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 41
  9. 2021Neural Response Interpretation through the Lens of Critical Pathwayscited by 32
  10. 2023Don't trust your eyes: on the (un)reliability of feature visualizationscited by 32
  11. 2022What Makes a Good Explanation?: A Harmonized View of Properties of Explanationscited by 21
  12. 2022Do Explanations Explain? Model Knows Bestcited by 20
38 more
  1. 2021Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Informationcited by 17
  2. 2022ExSum: From Local Explanations to Model Understandingcited by 15
  3. 2022A Fine-grained Interpretability Evaluation Benchmark for Neural NLPcited by 15
  4. 2024Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metricscited by 15
  5. 2022On the Relationship Between Explanation and Prediction: A Causal Viewcited by 14
  6. 2022Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methodscited by 12
  7. 2022The Solvability of Interpretability Evaluation Metricscited by 11
  8. 2022Explainable AI for Bioinformatics: Methods, Tools, and Applicationscited by 11
  9. 2023Understanding and Unifying Fourteen Attribution Methods with Taylor Interactionscited by 11
  10. 2021Explaining COVID-19 and Thoracic Pathology Model Predictions by Identifying Informative Input Featurescited by 9
  11. 2023Pixel-Level Explanation of Multiple Instance Learning Models in Biomedical Single Cell Imagescited by 9
  12. 2021Contrastive Counterfactual Visual Explanations With Overdeterminationcited by 7
  13. 2024Provably Better Explanations with Optimized Aggregation of Feature Attributionscited by 6
  14. 2025Uncertainty Propagation in XAI: A Comparison of Analytical and Empirical Estimatorscited by 6
  15. 2021Believe The HiPe: Hierarchical Perturbation for Fast, Robust, and Model-Agnostic Saliency Mappingcited by 5
  16. 2022A Functional Information Perspective on Model Interpretationcited by 5
  17. 2021From Heatmaps to Structural Explanations of Image Classifierscited by 4
  18. 2021Mutual Information Preserving Back-propagation: Learn to Invert for Faithful Attributioncited by 3
  19. 2021Evaluating saliency methods on artificial data with different background typescited by 3
  20. 2022Training Characteristic Functions with Reinforcement Learning: XAI-methods play Connect Fourcited by 3
  21. 2022Visualizing Deep Neural Networks with Topographic Activation Mapscited by 3
  22. 2022A Rigorous Study Of The Deep Taylor Decompositioncited by 3
  23. 2023Robustness of Visual Explanations to Common Data Augmentationcited by 3
  24. 2024VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretabilitycited by 3
  25. 2023Integrating Eye-Gaze Data into CXR DL Approaches: A Preliminary studycited by 2
  26. 2023How good Neural Networks interpretation methods really are? A quantitative benchmarkcited by 2
  27. 2023On the stability, correctness and plausibility of visual explanation methods based on feature importancecited by 2
  28. 2024CoProNN: Concept-based Prototypical Nearest Neighbors for Explaining Vision Modelscited by 2
  29. 2024Are Objective Explanatory Evaluation metrics Trustworthy? An Adversarial Analysiscited by 2
  30. 2024AdaptGrad: Adaptive Sampling to Reduce Noisecited by 2
  31. 2025Attribution Explanations for Deep Neural Networks: A Theoretical Perspectivecited by 2
  32. 2021Two4Two: Evaluating Interpretable Machine Learning - A Synthetic Dataset For Controlled Experimentscited by 1
  33. 2021SizeNet: Object Recognition via Object Real Size-based Convolutional Networkscited by 1
  34. 2021A Comparison of Deep Saliency Map Generators on Multispectral Data in Object Detectioncited by 1
  35. 2023On The Coherence of Quantitative Evaluation of Visual Explanationscited by 1
  36. 2023The Representational Status of Deep Learning Modelscited by 1
  37. 2021Attribution Mask: Filtering Out Irrelevant Features By Recursively Focusing Attention on Inputs of DNNscited by 0
  38. 2021A General Taylor Framework for Unifying and Revisiting Attribution Methodscited by 0

50 most cited of 59 All 59 (JSON)