# Reasoning about Entailment with Neural Attention

arXiv: https://arxiv.org/abs/1509.06664 · HTML: https://mlsanity.com/paper/1509.06664/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2015.

## Built on by (most cited first, 30 of 69)

- [RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction](https://mlsanity.com/paper/2203.09101/) (2022)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning](https://mlsanity.com/paper/2104.04697/) (2021)
- [On the Explainability of Natural Language Processing Deep Models](https://mlsanity.com/paper/2210.06929/) (2022)
- [Validating Large Language Models with ReLM](https://mlsanity.com/paper/2211.15458/) (2022)
- [Revisiting Large Language Models as Zero-shot Relation Extractors](https://mlsanity.com/paper/2310.05028/) (2023)
- [Dodrio: Exploring Transformer Models with Interactive Visualization](https://mlsanity.com/paper/2103.14625/) (2021)
- [Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization](https://mlsanity.com/paper/2110.10030/) (2021)
- [Time-aware Multiway Adaptive Fusion Network for Temporal Knowledge Graph Question Answering](https://mlsanity.com/paper/2302.12529/) (2023)
- [Question Calibration and Multi-Hop Modeling for Temporal Question Answering](https://mlsanity.com/paper/2402.13188/) (2024)
- [LOT: A Story-Centric Benchmark for Evaluating Chinese Long Text Understanding and Generation](https://mlsanity.com/paper/2108.12960/) (2021)
- [Attention Can Reflect Syntactic Structure (If You Let It)](https://mlsanity.com/paper/2101.10927/) (2021)
- [Reinforced Video Captioning with Entailment Rewards](https://mlsanity.com/paper/1708.02300/) (2017)
- [WikiContradiction: Detecting Self-Contradiction Articles on Wikipedia](https://mlsanity.com/paper/2111.08543/) (2021)
- [Neural ranking models for document retrieval](https://mlsanity.com/paper/2102.11903/) (2021)
- [Zero-shot Transfer of Article-aware Legal Outcome Classification for European Court of Human Rights Cases](https://mlsanity.com/paper/2302.00609/) (2023)
- [Granger-causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks](https://mlsanity.com/paper/1802.02195/) (2018)
- [Paraphrase Identification with Deep Learning: A Review of Datasets and Methods](https://mlsanity.com/paper/2212.06933/) (2022)
- [Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks](https://mlsanity.com/paper/2104.04488/) (2021)
- [Sentence Similarity Based on Contexts](https://mlsanity.com/paper/2105.07623/) (2021)
- [GLiREL -- Generalist Model for Zero-Shot Relation Extraction](https://mlsanity.com/paper/2501.03172/) (2025)
- [Resolving Word Vagueness with Scenario-guided Adapter for Natural Language Inference](https://mlsanity.com/paper/2405.12434/) (2024)
- [Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Prompting](https://mlsanity.com/paper/2410.01154/) (2024)
- [An Explainable CNN Approach for Medical Codes Prediction from Clinical Text](https://mlsanity.com/paper/2101.11430/) (2021)
- [Prompt-based Zero-shot Relation Extraction with Semantic Knowledge Augmentation](https://mlsanity.com/paper/2112.04539/) (2021)
- [Neural Attention Models in Deep Learning: Survey and Taxonomy](https://mlsanity.com/paper/2112.05909/) (2021)
- [Are the Multilingual Models Better? Improving Czech Sentiment with Transformers](https://mlsanity.com/paper/2108.10640/) (2021)
- [Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translation](https://mlsanity.com/paper/2505.08546/) (2025)
- [Set-to-Sequence Methods in Machine Learning: a Review](https://mlsanity.com/paper/2103.09656/) (2021)
- [Face Trees for Expression Recognition](https://mlsanity.com/paper/2112.02487/) (2021)

## Abstract (arXiv)

While most approaches to automatically recognizing entailment relations have used classifiers employing hand engineered features derived from complex natural language processing pipelines, in practice their performance has been only slightly better than bag-of-word pair classifiers using only lexical similarity. The only attempt so far to build an end-to-end differentiable neural network for entailment failed to outperform such a simple similarity classifier. In this paper, we propose a neural model that reads two sentences to determine entailment using long short-term memory units. We extend this model with a word-by-word neural attention mechanism that encourages reasoning over entailments of pairs of words and phrases. Furthermore, we present a qualitative analysis of attention weights produced by this model, demonstrating such reasoning capabilities. On a large entailment dataset this model outperforms the previous best neural model and a classifier with engineered features by a substantial margin. It is the first generic end-to-end differentiable system that achieves state-of-the-art accuracy on a textual entailment dataset.