# Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference

arXiv: https://arxiv.org/abs/1808.03894 · HTML: https://mlsanity.com/paper/1808.03894/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2018.

## Built on by (most cited first, 26 of 26)

- [Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond](https://mlsanity.com/paper/2103.10689/) (2021)
- [Visual correspondence-based explanations improve AI robustness and human-AI team accuracy](https://mlsanity.com/paper/2208.00780/) (2022)
- [Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification](https://mlsanity.com/paper/2105.02657/) (2021)
- [Deconfounding Legal Judgment Prediction for European Court of Human Rights Cases Towards Better Alignment with Experts](https://mlsanity.com/paper/2210.13836/) (2022)
- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [Fooling Explanations in Text Classifiers](https://mlsanity.com/paper/2206.03178/) (2022)
- [Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks](https://mlsanity.com/paper/2104.04488/) (2021)
- [More Interpretable Graph Similarity Computation via Maximum Common Subgraph Inference](https://mlsanity.com/paper/2208.04580/) (2022)
- [SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning](https://mlsanity.com/paper/2104.08793/) (2021)
- [Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning](https://mlsanity.com/paper/2103.03446/) (2021)
- [Faithful and Accurate Self-Attention Attribution for Message Passing Neural Networks via the Computation Tree Viewpoint](https://mlsanity.com/paper/2406.04612/) (2024)
- [Controlling the Focus of Pretrained Language Generation Models](https://mlsanity.com/paper/2203.01146/) (2022)
- [Fake News Detection After LLM Laundering: Measurement and Explanation](https://mlsanity.com/paper/2501.18649/) (2025)
- [LioNets: A Neural-Specific Local Interpretation Technique Exploiting Penultimate Layer Information](https://mlsanity.com/paper/2104.06057/) (2021)
- [Do Models Learn the Directionality of Relations? A New Evaluation: Relation Direction Recognition](https://mlsanity.com/paper/2105.09045/) (2021)
- [Reason induced visual attention for explainable autonomous driving](https://mlsanity.com/paper/2110.07380/) (2021)
- [Identifying the Source of Vulnerability in Explanation Discrepancy: A Case Study in Neural Text Classification](https://mlsanity.com/paper/2212.05327/) (2022)
- [Towards Reconciling Usability and Usefulness of Explainable AI Methodologies](https://mlsanity.com/paper/2301.05347/) (2023)
- [On Explaining with Attention Matrices](https://mlsanity.com/paper/2410.18541/) (2024)
- [POTATO: exPlainable infOrmation exTrAcTion framewOrk](https://mlsanity.com/paper/2201.13230/) (2022)
- [A Study of the Plausibility of Attention between RNN Encoders in Natural Language Inference](https://mlsanity.com/paper/2501.13735/) (2025)
- [Toward a traceable, explainable, and fairJD/Resume recommendation system](https://mlsanity.com/paper/2202.08960/) (2022)
- [STRATA: Word Boundaries & Phoneme Recognition From Continuous Urdu Speech using Transfer Learning, Attention, & Data Augmentation](https://mlsanity.com/paper/2204.07848/) (2022)
- [Using Interpretation Methods for Model Enhancement](https://mlsanity.com/paper/2404.02068/) (2024)
- [Hierarchical Attention Network for Interpretable ECG-based Heart Disease Classification](https://mlsanity.com/paper/2504.03703/) (2025)

## Abstract (arXiv)

Deep learning models have achieved remarkable success in natural language inference (NLI) tasks. While these models are widely explored, they are hard to interpret and it is often unclear how and why they actually work. In this paper, we take a step toward explaining such deep learning based models through a case study on a popular neural model for NLI. In particular, we propose to interpret the intermediate layers of NLI models by visualizing the saliency of attention and LSTM gating signals. We present several examples for which our methods are able to reveal interesting insights and identify the critical information contributing to the model decisions.