Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference
Not on the map. Why?
Abstract (arXiv)
Deep learning models have achieved remarkable success in natural language inference (NLI) tasks. While these models are widely explored, they are hard to interpret and it is often unclear how and why they actually work. In this paper, we take a step toward explaining such deep learning based models through a case study on a popular neural model for NLI. In particular, we propose to interpret the intermediate layers of NLI models by visualizing the saliency of attention and LSTM gating signals. We present several examples for which our methods are able to reveal interesting insights and identify the critical information contributing to the model decisions.
Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Aug 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 9; 2022: 9; 2023: 1; 2024: 3; 2025: 4.
26 papers.
Sort:
- 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
- 2022Visual correspondence-based explanations improve AI robustness and human-AI team accuracycited by 37
- 2021Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classificationcited by 29
- 2022Deconfounding Legal Judgment Prediction for European Court of Human Rights Cases Towards Better Alignment with Expertscited by 24
- 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
- 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
- 2022Fooling Explanations in Text Classifierscited by 13
- 2021Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Maskscited by 12
- 2022More Interpretable Graph Similarity Computation via Maximum Common Subgraph Inferencecited by 8
- 2021SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoningcited by 7
- 2021Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learningcited by 5
- 2024Faithful and Accurate Self-Attention Attribution for Message Passing Neural Networks via the Computation Tree Viewpointcited by 5
14 more
- 2022Controlling the Focus of Pretrained Language Generation Modelscited by 4
- 2025Fake News Detection After LLM Laundering: Measurement and Explanationcited by 4
- 2021LioNets: A Neural-Specific Local Interpretation Technique Exploiting Penultimate Layer Informationcited by 3
- 2021Do Models Learn the Directionality of Relations? A New Evaluation: Relation Direction Recognitioncited by 2
- 2021Reason induced visual attention for explainable autonomous drivingcited by 2
- 2022Identifying the Source of Vulnerability in Explanation Discrepancy: A Case Study in Neural Text Classificationcited by 2
- 2023Towards Reconciling Usability and Usefulness of Explainable AI Methodologiescited by 2
- 2022POTATO: exPlainable infOrmation exTrAcTion framewOrkcited by 1
- 2024On Explaining with Attention Matricescited by 1
- 2025A Study of the Plausibility of Attention between RNN Encoders in Natural Language Inferencecited by 1
- 2022Toward a traceable, explainable, and fairJD/Resume recommendation systemcited by 0
- 2022STRATA: Word Boundaries & Phoneme Recognition From Continuous Urdu Speech using Transfer Learning, Attention, & Data Augmentationcited by 0
- 2024Using Interpretation Methods for Model Enhancementcited by 0
- 2025Hierarchical Attention Network for Interpretable ECG-based Heart Disease Classificationcited by 0
- 2025Hierarchical Attention Network for Interpretable ECG-based Heart Disease Classification25 Mar 2025
- 2025Fake News Detection After LLM Laundering: Measurement and Explanation29 Jan 2025
- 2025A Comprehensive Survey on Self-Interpretable Neural Networks26 Jan 2025
- 2025A Study of the Plausibility of Attention between RNN Encoders in Natural Language Inference23 Jan 2025
- 2024On Explaining with Attention Matrices24 Oct 2024
- 2024Faithful and Accurate Self-Attention Attribution for Message Passing Neural Networks via the Computation Tree Viewpoint7 Jun 2024
- 2024Using Interpretation Methods for Model Enhancement2 Apr 2024
- 2023Towards Reconciling Usability and Usefulness of Explainable AI Methodologies13 Jan 2023
- 2022Identifying the Source of Vulnerability in Explanation Discrepancy: A Case Study in Neural Text Classification10 Dec 2022
- 2022Deconfounding Legal Judgment Prediction for European Court of Human Rights Cases Towards Better Alignment with Experts25 Oct 2022
- 2022More Interpretable Graph Similarity Computation via Maximum Common Subgraph Inference9 Aug 2022
- 2022Visual correspondence-based explanations improve AI robustness and human-AI team accuracy26 Jul 2022
14 more
- 2022Fooling Explanations in Text Classifiers7 Jun 2022
- 2022STRATA: Word Boundaries & Phoneme Recognition From Continuous Urdu Speech using Transfer Learning, Attention, & Data Augmentation16 Apr 2022
- 2022Controlling the Focus of Pretrained Language Generation Models2 Mar 2022
- 2022Toward a traceable, explainable, and fairJD/Resume recommendation system2 Feb 2022
- 2022POTATO: exPlainable infOrmation exTrAcTion framewOrk31 Jan 2022
- 2021Interpreting Deep Learning Models in Natural Language Processing: A Review20 Oct 2021
- 2021Reason induced visual attention for explainable autonomous driving11 Oct 2021
- 2021Do Models Learn the Directionality of Relations? A New Evaluation: Relation Direction Recognition19 May 2021
- 2021Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification6 May 2021
- 2021SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning18 Apr 2021
- 2021LioNets: A Neural-Specific Local Interpretation Technique Exploiting Penultimate Layer Information13 Apr 2021
- 2021Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks9 Apr 2021
- 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond19 Mar 2021
- 2021Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning5 Mar 2021
- 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
- 2022Visual correspondence-based explanations improve AI robustness and human-AI team accuracycited by 37
- 2021Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classificationcited by 29
- 2022Deconfounding Legal Judgment Prediction for European Court of Human Rights Cases Towards Better Alignment with Expertscited by 24
- 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
- 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
- 2022Fooling Explanations in Text Classifierscited by 13
- 2021Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Maskscited by 12
- 2022More Interpretable Graph Similarity Computation via Maximum Common Subgraph Inferencecited by 8
- 2021SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoningcited by 7
- 2021Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learningcited by 5
- 2024Faithful and Accurate Self-Attention Attribution for Message Passing Neural Networks via the Computation Tree Viewpointcited by 5
12 most cited of 26 All 26 (JSON)
- 2025Hierarchical Attention Network for Interpretable ECG-based Heart Disease Classification25 Mar 2025
- 2025Fake News Detection After LLM Laundering: Measurement and Explanation29 Jan 2025
- 2025A Comprehensive Survey on Self-Interpretable Neural Networks26 Jan 2025
- 2025A Study of the Plausibility of Attention between RNN Encoders in Natural Language Inference23 Jan 2025
- 2024On Explaining with Attention Matrices24 Oct 2024
- 2024Faithful and Accurate Self-Attention Attribution for Message Passing Neural Networks via the Computation Tree Viewpoint7 Jun 2024
- 2024Using Interpretation Methods for Model Enhancement2 Apr 2024
- 2023Towards Reconciling Usability and Usefulness of Explainable AI Methodologies13 Jan 2023
- 2022Identifying the Source of Vulnerability in Explanation Discrepancy: A Case Study in Neural Text Classification10 Dec 2022
- 2022Deconfounding Legal Judgment Prediction for European Court of Human Rights Cases Towards Better Alignment with Experts25 Oct 2022
- 2022More Interpretable Graph Similarity Computation via Maximum Common Subgraph Inference9 Aug 2022
- 2022Visual correspondence-based explanations improve AI robustness and human-AI team accuracy26 Jul 2022
12 newest of 26 All 26 (JSON)