# Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformers

arXiv: https://arxiv.org/abs/2607.10365 · HTML: https://mlsanity.com/paper/2607.10365/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 11 Jul 2026.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [Full-Gradient Representation for Neural Network Visualization](https://mlsanity.com/paper/1905.00780/) (2019)
- [Self-Attention Attribution: Interpreting Information Interactions Inside Transformer](https://mlsanity.com/paper/2004.11207/) (2020)
- [AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers](https://mlsanity.com/paper/2402.05602/) (2024)
- [XAI for Transformers: Better Explanations through Conservative Propagation](https://mlsanity.com/paper/2202.07304/) (2022)
- [Learning to Estimate Shapley Values with Vision Transformers](https://mlsanity.com/paper/2206.05282/) (2022)
- [Rethinking Attention-Model Explainability through Faithfulness Violation Test](https://mlsanity.com/paper/2201.12114/) (2022)
- [Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps](https://mlsanity.com/paper/2204.11073/) (2022)
- [ViT-CX: Causal Explanation of Vision Transformers](https://mlsanity.com/paper/2211.03064/) (2022)
- [Finer-CAM: Spotting the Difference Reveals Finer Details for Visual Explanation](https://mlsanity.com/paper/2501.11309/) (2025)
- [Attention mechanisms and deep learning for machine vision: A survey of the state of the art](https://mlsanity.com/paper/2106.07550/) (2021)
- [Layer-Wise Relevance Propagation with Conservation Property for ResNet](https://mlsanity.com/paper/2407.09115/) (2024)
- [Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability](https://mlsanity.com/paper/2506.02138/) (2025)

## Abstract (arXiv)

Vision Transformers (ViTs) are difficult to interpret because current methods of relevance propagation and attention flow do not fully consider some key architectural features, such as the uneven importance of attention heads and residual connections. Prior approaches typically assume uniform importance across attention heads; furthermore, they model skip connections as identity paths, leading to inaccurate relevance attribution. To address these issues, we introduce GradSkip, a novel relevance propagation method for ViTs based on adaptive head weighting and skip-aware propagation. GradSkip models the different importance of the attention heads and dynamically distributes relevance between the attention and residual paths. Experiments on ImageNet1K and BloodMNIST demonstrate a state-of-the-art faithfulness of GradSkip while requiring over 14 times fewer GFLOPs than the best-performing existing approaches. Additional evaluations using transformer-based segmentation confirm improved localization and alignment with ground-truth regions.