# Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks

arXiv: https://arxiv.org/abs/1904.00605 · HTML: https://mlsanity.com/paper/1904.00605/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2019.

## Built on by (most cited first, 30 of 49)

- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond](https://mlsanity.com/paper/2103.10689/) (2021)
- [The Hidden Attention of Mamba Models](https://mlsanity.com/paper/2403.01590/) (2024)
- [Optimizing Relevance Maps of Vision Transformers Improves Robustness](https://mlsanity.com/paper/2206.01161/) (2022)
- [Focus! Rating XAI Methods and Finding Biases](https://mlsanity.com/paper/2109.15035/) (2021)
- [On the Faithfulness of Vision Transformer Explanations](https://mlsanity.com/paper/2404.01415/) (2024)
- [Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoring](https://mlsanity.com/paper/2102.07805/) (2021)
- [Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixing](https://mlsanity.com/paper/2112.08796/) (2021)
- [Debiasing Deep Chest X-Ray Classifiers using Intra- and Post-processing Methods](https://mlsanity.com/paper/2208.00781/) (2022)
- [Explanations for Occluded Images](https://mlsanity.com/paper/2103.03622/) (2021)
- [Latent Space Explanation by Intervention](https://mlsanity.com/paper/2112.04895/) (2021)
- [Explaining Modern Gated-Linear RNNs via a Unified Implicit Attention Formulation](https://mlsanity.com/paper/2405.16504/) (2024)
- [Towards Modeling Uncertainties of Self-explaining Neural Networks via Conformal Prediction](https://mlsanity.com/paper/2401.01549/) (2024)
- [From Modern CNNs to Vision Transformers: Assessing the Performance, Robustness, and Classification Strategies of Deep Learning Models in Histopathology](https://mlsanity.com/paper/2204.05044/) (2022)
- [Grad-ECLIP: Gradient-based Visual and Textual Explanations for CLIP](https://mlsanity.com/paper/2502.18816/) (2025)
- [Disentangling Structure and Style: Political Bias Detection in News by Inducing Document Hierarchy](https://mlsanity.com/paper/2304.02247/) (2023)
- [Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groups](https://mlsanity.com/paper/2310.16316/) (2023)
- [MISA: Online Defense of Trojaned Models using Misattributions](https://mlsanity.com/paper/2103.15918/) (2021)
- [Explanatory Pluralism in Explainable AI](https://mlsanity.com/paper/2106.13976/) (2021)
- [Multiple Different Black Box Explanations for Image Classifiers](https://mlsanity.com/paper/2309.14309/) (2023)
- [Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models](https://mlsanity.com/paper/2310.16584/) (2023)
- [Finding Representative Interpretations on Convolutional Neural Networks](https://mlsanity.com/paper/2108.06384/) (2021)
- [It Ain't That Bad: Understanding the Mysterious Performance Drop in OOD Generalization for Generative Transformer Models](https://mlsanity.com/paper/2308.08268/) (2023)
- [SPADE: Sparsity-Guided Debugging for Deep Neural Networks](https://mlsanity.com/paper/2310.04519/) (2023)
- [How Well Do Feature-Additive Explainers Explain Feature-Additive Predictors?](https://mlsanity.com/paper/2310.18496/) (2023)
- [You Only Explain Once](https://mlsanity.com/paper/2311.14081/) (2023)
- [Towards Fine-Grained Interpretability: Counterfactual Explanations for Misclassification with Saliency Partition](https://mlsanity.com/paper/2511.07974/) (2025)
- [NEWRON: A New Generalization of the Artificial Neuron to Enhance the Interpretability of Neural Networks](https://mlsanity.com/paper/2110.02775/) (2021)
- [Learning a Weight Map for Weakly-Supervised Localization](https://mlsanity.com/paper/2111.14131/) (2021)

## Abstract (arXiv)

As Deep Neural Networks (DNNs) have demonstrated superhuman performance in a variety of fields, there is an increasing interest in understanding the complex internal mechanisms of DNNs. In this paper, we propose Relative Attributing Propagation (RAP), which decomposes the output predictions of DNNs with a new perspective of separating the relevant (positive) and irrelevant (negative) attributions according to the relative influence between the layers. The relevance of each neuron is identified with respect to its degree of contribution, separated into positive and negative, while preserving the conservation rule. Considering the relevance assigned to neurons in terms of relative priority, RAP allows each neuron to be assigned with a bi-polar importance score concerning the output: from highly relevant to highly irrelevant. Therefore, our method makes it possible to interpret DNNs with much clearer and attentive visualizations of the separated attributions than the conventional explaining methods. To verify that the attributions propagated by RAP correctly account for each meaning, we utilize the evaluation metrics: (i) Outside-inside relevance ratio, (ii) Segmentation mIOU and (iii) Region perturbation. In all experiments and metrics, we present a sizable gap in comparison to the existing literature. Our source code is available in urlhttps://github.com/wjNam/RelativeAttributingPropagation.