mlsanityData to 7 Oct 2026

Full-Gradient Representation for Neural Network Visualization

arXiv 1905.00780 ↗First posted 2 May 2019cs.LG cs.CV stat.MLcited by 189

Not on the map. Why?

Abstract (arXiv)

We introduce a new tool for interpreting neural net responses, namely full-gradients, which decomposes the neural net response into input sensitivity and per-neuron sensitivity components. This is the first proposed representation which satisfies two key properties: completeness and weak dependence, which provably cannot be satisfied by any saliency map-based interpretability method. For convolutional nets, we also propose an approximate saliency map representation, called FullGrad, obtained by aggregating the full-gradient components. We experimentally evaluate the usefulness of FullGrad in explaining model behaviour with two quantitative tests: pixel perturbation and remove-and-retrain. Our experiments reveal that our method explains model behaviour correctly, and more comprehensively than other methods in the literature. Visual inspection also reveals that our saliency maps are sharper and more tightly confined to object regions than other methods.

Key facts. Cited by 189 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 31; 2023: 33; 2024: 39; 2025: 31; 2026: 27.

189 papers.

Sort:

By area:

  1. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 316
  2. 2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
  3. 2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 124
  4. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  5. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  6. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  7. 2022Diffusion Visual Counterfactual Explanationscited by 96
  8. 2022B-cos Networks: Alignment is All We Need for Interpretabilitycited by 88
  9. 2022Impossibility Theorems for Feature Attributioncited by 78
  10. 2022Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanationscited by 77
  11. 2022MaskTune: Mitigating Spurious Correlations by Forcing to Explorecited by 66
  12. 2021Synthetic Benchmarks for Scientific Research in Explainable Machine Learningcited by 59
38 more
  1. 2021Convolutional Dynamic Alignment Networks for Interpretable Classificationscited by 54
  2. 2021Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networkscited by 48
  3. 2021Shapley Explanation Networkscited by 41
  4. 2022Optimizing Relevance Maps of Vision Transformers Improves Robustnesscited by 41
  5. 2024Denoising: A Powerful Building-Block for Imaging, Inverse Problems, and Machine Learningcited by 35
  6. 2024Gradient based Feature Attribution in Explainable AI: A Technical Reviewcited by 34
  7. 2023Don't trust your eyes: on the (un)reliability of feature visualizationscited by 32
  8. 2023B-cos Alignment for Inherently Interpretable CNNs and Vision Transformerscited by 31
  9. 2022Sparse Visual Counterfactual Explanations in Image Spacecited by 30
  10. 2022Towards Better Understanding Attribution Methodscited by 28
  11. 2022Visualizing and Understanding Patch Interactions in Vision Transformercited by 26
  12. 2022Discovering Transferable Forensic Features for CNN-generated Images Detectioncited by 24
  13. 2021Vision Transformer using Low-level Chest X-ray Feature Corpus for COVID-19 Diagnosis and Severity Quantificationcited by 21
  14. 2022Do Explanations Explain? Model Knows Bestcited by 20
  15. 2022Self-Supervised Learning for Invariant Representations from Multi-Spectral and SAR Imagescited by 20
  16. 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
  17. 2022Custom Structure Preservation in Face Agingcited by 19
  18. 2024Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformercited by 19
  19. 2024On the Faithfulness of Vision Transformer Explanationscited by 19
  20. 2021Consistent Explanations by Contrastive Learningcited by 18
  21. 2023Visual Explanations via Iterated Integrated Attributionscited by 18
  22. 2021Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoringcited by 16
  23. 2022Attribution-based XAI Methods in Computer Vision: A Reviewcited by 16
  24. 2021Keep CALM and Improve Visual Feature Attributioncited by 15
  25. 2022Debiasing Deep Chest X-Ray Classifiers using Intra- and Post-processing Methodscited by 15
  26. 2021Saliency Guided Experience Packing for Replay in Continual Learningcited by 14
  27. 2024All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregationcited by 14
  28. 2021Latent Space Explanation by Interventioncited by 13
  29. 2022Backdoor Attacks on Vision Transformerscited by 13
  30. 2021PACE: Posthoc Architecture-Agnostic Concept Extractor for Explaining CNNscited by 12
  31. 2023Semantic Adversarial Attacks via Diffusion Modelscited by 12
  32. 2024B-cosification: Transforming Deep Neural Networks to be Inherently Interpretablecited by 12
  33. 2025Show and Tell: Visually Explainable Deep Neural Nets via Spatially-Aware Concept Bottleneck Modelscited by 12
  34. 2021Ada-SISE: Adaptive Semantic Input Sampling for Efficient Explanation of Convolutional Neural Networkscited by 11
  35. 2023SketchXAI: A First Look at Explainability for Human Sketchescited by 10
  36. 2023Discriminative Feature Attributions: Bridging Post Hoc Explainability and Inherent Interpretabilitycited by 10
  37. 2024Regulating Model Reliance on Non-Robust Features by Smoothing Input Marginal Densitycited by 10
  38. 2025Winsor-CAM: Human-Tunable Visual Explanations from Deep Networks via Layer-Wise Winsorizationcited by 10

50 most cited of 189 All 189 (JSON)