Explaining Recurrent Neural Network Predictions in Sentiment Analysis
Not on the map. Why?
Abstract (arXiv)
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specific propagation rule applicable to multiplicative connections as they arise in recurrent network architectures such as LSTMs and GRUs. We apply our technique to a word-based bi-directional LSTM model on a five-class sentiment prediction task, and evaluate the resulting LRP relevances both qualitatively and quantitatively, obtaining better results than a gradient-based related method which was used in previous work.
Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jun 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 26; 2022: 18; 2023: 15; 2024: 14; 2025: 5; 2026: 9.
87 papers.
Sort:
By area:
- 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
- 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
- 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
- 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
- 2024The Hidden Attention of Mamba Modelscited by 88
- 2021Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysiscited by 65
- 2021On Guaranteed Optimal Robust Explanations for NLP Modelscited by 54
- 2021Evaluating Saliency Methods for Neural Language Modelscited by 37
- 2021Explainable AI: current status and future directionscited by 32
- 2021Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classificationcited by 29
- 2022Diffeomorphic Counterfactuals with Generative Modelscited by 29
- 2022Human Interpretation of Saliency-based Explanation Over Textcited by 25
38 more
- 2022Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunitiescited by 25
- 2022Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspacescited by 25
- 2021On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classificationcited by 21
- 2021Levels of explainable artificial intelligence for human-aligned conversational explanationscited by 21
- 2022Learning to Scaffold: Optimizing Model Explanations for Teachingcited by 20
- 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
- 2021Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Saliencecited by 16
- 2023Interpretation of Time-Series Deep Models: A Surveycited by 15
- 2021Model Explainability in Deep Learning Based Natural Language Processingcited by 14
- 2023Being Right for Whose Right Reasons?cited by 12
- 2021Thermostat: A Large Collection of NLP Model Explanations and Analysis Toolscited by 11
- 2022Explainable AI for Bioinformatics: Methods, Tools, and Applicationscited by 11
- 2023Don't be fooled: label leakage in explanation methods and the importance of their quantitative evaluationcited by 11
- 2021Flexible Instance-Specific Rationalization of NLP Modelscited by 10
- 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 10
- 2022How explainable are adversarially-robust CNNs?cited by 9
- 2024Explaining Text Similarity in Transformer Modelscited by 9
- 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
- 2022Visualizing and Understanding Contrastive Learningcited by 7
- 2022Explaining Image Classifiers with Multiscale Directional Image Representationcited by 7
- 2024Layer-Wise Relevance Propagation with Conservation Property for ResNetcited by 7
- 2021Where do Models go Wrong? Parameter-Space Saliency Maps for Explainabilitycited by 6
- 2024SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generationcited by 6
- 2025Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainabilitycited by 6
- 2021Explainability: Relevance based Dynamic Deep Learning Algorithm for Fault Detection and Diagnosis in Chemical Processescited by 5
- 2021Double Trouble: How to not explain a text classifier's decisions using counterfactuals synthesized by masked language models?cited by 5
- 2022Explainable Online Lane Change Predictions on a Digital Twin with a Layer Normalized LSTM and Layer-wise Relevance Propagationcited by 5
- 2022ReX: A Framework for Incorporating Temporal Information in Model-Agnostic Local Explanation Techniquescited by 5
- 2022Feature Importance for Time Series Data: Improving KernelSHAPcited by 5
- 2023Decoding Layer Saliency in Language Transformerscited by 5
- 2022Don't Get Me Wrong: How to Apply Deep Visual Interpretations to Time Seriescited by 4
- 2022Implicit N-grams Induced by Recurrencecited by 4
- 2022Explainable Artificial Intelligence: Precepts, Methods, and Opportunities for Research in Constructioncited by 4
- 2024SIDU-TXT: An XAI Algorithm for NLP with a Holistic Assessment Approachcited by 4
- 2021On the Post-hoc Explainability of Deep Echo State Networks for Time Series Forecasting, Image and Video Classificationcited by 3
- 2021Do End-to-End Speech Recognition Models Care About Context?cited by 3
- 2021Evaluating deep transfer learning for whole-brain cognitive decodingcited by 3
- 2022A Rigorous Study Of The Deep Taylor Decompositioncited by 3
50 most cited of 87 All 87 (JSON)
- 2026Explaining Hyperbolic Neural Networks via Geometry-Aware Relevance Propagation28 Sep 2026
- 2026Local Additive Feature Attribution: A Mathematical Taxonomy and Reporting Checklist15 Jul 2026
- 2026Explaining Temporal Graph Neural Networks via Feature-induced Information Flow25 Jun 2026
- 2026Concept-based explanations of Segmentation and Detection models in Natural Disaster Management24 Mar 2026
- 2026Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models18 Feb 2026
- 2026HateXScore: A Metric Suite for Evaluating Reasoning Quality in Hate Speech Explanations20 Jan 2026
- 2026Reasoning Stabilization Point: A Training-Time Signal for Stable Evidence and Shortcut Reliance12 Jan 2026
- 2026CRANE: Causal Relevance Analysis of Language-Specific Neurons in Multilingual Large Language Models8 Jan 2026
- 2026Prompting Underestimates LLM Capability for Time Series Classification6 Jan 2026
- 2025The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models30 Sep 2025
- 2025Attention Lattice Adapter: Visual Explanation Generation for Visual Foundation Model18 Sep 2025
- 2025Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability2 Jun 2025
38 more
- 2025Towards an Evaluation Framework for Explainable Artificial Intelligence Systems for Health and Well-being11 Apr 2025
- 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Models21 Feb 2025
- 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future18 Dec 2024
- 2024Advancing Attribution-Based Neural Network Explainability through Relative Absolute Magnitude Layer-Wise Relevance Propagation and Multi-Component Evaluation12 Dec 2024
- 2024LossVal: Efficient Data Valuation for Neural Networks5 Dec 2024
- 2024SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generation3 Nov 2024
- 2024Explanation sensitivity to the randomness of large language models: the case of journalistic text classification7 Oct 2024
- 2024What Matters in Explanations: Towards Explainable Fake Review Detection Focusing on Transformers24 Jul 2024
- 2024Layer-Wise Relevance Propagation with Conservation Property for ResNet12 Jul 2024
- 2024Explaining Text Similarity in Transformer Models10 May 2024
- 2024On the Road to Clarity: Exploring Explainable AI for World Models in a Driver Assistance System26 Apr 2024
- 2024Sparse Explanations of Neural Networks Using Pruned Layer-Wise Relevance Propagation22 Apr 2024
- 2024Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models11 Apr 2024
- 2024The Hidden Attention of Mamba Models3 Mar 2024
- 2024SIDU-TXT: An XAI Algorithm for NLP with a Holistic Assessment Approach5 Feb 2024
- 2024Beyond Fidelity: Explaining Vulnerability Localization of Learning-based Detectors5 Jan 2024
- 2023Interpretable and Explainable Machine Learning Methods for Predictive Process Monitoring: A Systematic Literature Review29 Dec 2023
- 2023A Review of Hybrid and Ensemble in Deep Learning for Natural Language Processing9 Dec 2023
- 2023Unveiling Black-boxes: Explainable Deep Learning Models for Patent Classification31 Oct 2023
- 2023Decoding Layer Saliency in Language Transformers9 Aug 2023
- 2023DARE: Towards Robust Text Explanations in Biomedical and Healthcare Applications5 Jul 2023
- 2023Being Right for Whose Right Reasons?1 Jun 2023
- 2023Interpretation of Time-Series Deep Models: A Survey23 May 2023
- 2023A Brief Review of Explainable Artificial Intelligence in Healthcare4 Apr 2023
- 2023Unlocking Layer-wise Relevance Propagation for Autoencoders21 Mar 2023
- 2023Don't be fooled: label leakage in explanation methods and the importance of their quantitative evaluation24 Feb 2023
- 2023Machine Learning based prediction of Glucose Levels in Type 1 Diabetes Patients with the use of Continuous Glucose Monitoring Data24 Feb 2023
- 2023Explaining text classifiers through progressive neighborhood approximation with realistic samples11 Feb 2023
- 2023A Biomedical Knowledge Graph for Biomarker Discovery in Cancer9 Feb 2023
- 2023Applications and Challenges of Sentiment Analysis in Real-life Scenarios24 Jan 2023
- 2023Explainability and Robustness of Deep Visual Classification Models3 Jan 2023
- 2022Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces30 Dec 2022
- 2022Explainable AI for Bioinformatics: Methods, Tools, and Applications25 Dec 2022
- 2022Explaining Image Classifiers with Multiscale Directional Image Representation22 Nov 2022
- 2022A Rigorous Study Of The Deep Taylor Decomposition14 Nov 2022
- 2022Explainable Artificial Intelligence: Precepts, Methods, and Opportunities for Research in Construction12 Nov 2022
- 2022Feature Importance for Time Series Data: Improving KernelSHAP5 Oct 2022
- 2022Towards Faithful Model Explanation in NLP: A Survey22 Sep 2022
50 newest of 87 All 87 (JSON)
- 2024The Hidden Attention of Mamba Modelscited by 88
- 2024The Hidden Attention of Mamba Models3 Mar 2024
- 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
- 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
- 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
- 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
- 2021Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysiscited by 65
- 2021On Guaranteed Optimal Robust Explanations for NLP Modelscited by 54
- 2021Evaluating Saliency Methods for Neural Language Modelscited by 37
- 2021Explainable AI: current status and future directionscited by 32
- 2021Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classificationcited by 29
- 2022Diffeomorphic Counterfactuals with Generative Modelscited by 29
- 2022Human Interpretation of Saliency-based Explanation Over Textcited by 25
- 2022Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunitiescited by 25
12 most cited of 84 All 84 (JSON)
- 2026Explaining Hyperbolic Neural Networks via Geometry-Aware Relevance Propagation28 Sep 2026
- 2026Local Additive Feature Attribution: A Mathematical Taxonomy and Reporting Checklist15 Jul 2026
- 2026Explaining Temporal Graph Neural Networks via Feature-induced Information Flow25 Jun 2026
- 2026Concept-based explanations of Segmentation and Detection models in Natural Disaster Management24 Mar 2026
- 2026Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models18 Feb 2026
- 2026HateXScore: A Metric Suite for Evaluating Reasoning Quality in Hate Speech Explanations20 Jan 2026
- 2026CRANE: Causal Relevance Analysis of Language-Specific Neurons in Multilingual Large Language Models8 Jan 2026
- 2026Prompting Underestimates LLM Capability for Time Series Classification6 Jan 2026
- 2025The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models30 Sep 2025
- 2025Attention Lattice Adapter: Visual Explanation Generation for Visual Foundation Model18 Sep 2025
- 2025Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability2 Jun 2025
- 2025Towards an Evaluation Framework for Explainable Artificial Intelligence Systems for Health and Well-being11 Apr 2025
12 newest of 84 All 84 (JSON)