# On Attribution of Recurrent Neural Network Predictions via Additive Decomposition

arXiv: https://arxiv.org/abs/1903.11245 · HTML: https://mlsanity.com/paper/1903.11245/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2019.

## Built on by (most cited first, 16 of 16)

- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Local Interpretations for Explainable Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.11072/) (2021)
- [Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks](https://mlsanity.com/paper/2104.04488/) (2021)
- [Developing a Fidelity Evaluation Approach for Interpretable Machine Learning](https://mlsanity.com/paper/2106.08492/) (2021)
- [An Attention Matrix for Every Decision: Faithfulness-based Arbitration Among Multiple Attention-Based Interpretations of Transformers in Text Classification](https://mlsanity.com/paper/2209.10876/) (2022)
- [Improving Interpretability via Explicit Word Interaction Graph Layer](https://mlsanity.com/paper/2302.02016/) (2023)
- [PromptExp: Multi-granularity Prompt Explanation of Large Language Models](https://mlsanity.com/paper/2410.13073/) (2024)
- [Step-resolved data attribution for looped transformers](https://mlsanity.com/paper/2602.10097/) (2026)
- [SIDU-TXT: An XAI Algorithm for NLP with a Holistic Assessment Approach](https://mlsanity.com/paper/2402.03043/) (2024)
- [LioNets: A Neural-Specific Local Interpretation Technique Exploiting Penultimate Layer Information](https://mlsanity.com/paper/2104.06057/) (2021)
- [On the Bias-Variance Characteristics of LIME and SHAP in High Sparsity Movie Recommendation Explanation Tasks](https://mlsanity.com/paper/2206.04784/) (2022)
- [Truthful Meta-Explanations for Local Interpretability of Machine Learning Models](https://mlsanity.com/paper/2212.03513/) (2022)
- [PEACH: Pretrained-embedding Explanation Across Contextual and Hierarchical Structure](https://mlsanity.com/paper/2404.13645/) (2024)
- [Beyond 5G Network Failure Classification for Network Digital Twin Using Graph Neural Network](https://mlsanity.com/paper/2406.06595/) (2024)
- [Responsible AI for General-Purpose Systems: Overview, Challenges, and A Path Forward](https://mlsanity.com/paper/2601.13122/) (2026)
- [READ: Reading the Selective Scan for Faithful Video Attribution](https://mlsanity.com/paper/2610.13776/) (2026)

## Abstract (arXiv)

RNN models have achieved the state-of-the-art performance in a wide range of text mining tasks. However, these models are often regarded as black-boxes and are criticized due to the lack of interpretability. In this paper, we enhance the interpretability of RNNs by providing interpretable rationales for RNN predictions. Nevertheless, interpreting RNNs is a challenging problem. Firstly, unlike existing methods that rely on local approximation, we aim to provide rationales that are more faithful to the decision making process of RNN models. Secondly, a flexible interpretation method should be able to assign contribution scores to text segments of varying lengths, instead of only to individual words. To tackle these challenges, we propose a novel attribution method, called REAT, to provide interpretations to RNN predictions. REAT decomposes the final prediction of a RNN into additive contribution of each word in the input text. This additive decomposition enables REAT to further obtain phrase-level attribution scores. In addition, REAT is generally applicable to various RNN architectures, including GRU, LSTM and their bidirectional versions. Experimental results demonstrate the faithfulness and interpretability of the proposed attribution method. Comprehensive analysis shows that our attribution method could unveil the useful linguistic knowledge captured by RNNs. Some analysis further demonstrates our method could be utilized as a debugging tool to examine the vulnerability and failure reasons of RNNs, which may lead to several promising future directions to promote generalization ability of RNNs.