# Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering

arXiv: https://arxiv.org/abs/1806.04330 · HTML: https://mlsanity.com/paper/1806.04330/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2018.

## Built on by (most cited first, 23 of 23)

- [On the Evaluation Metrics for Paraphrase Generation](https://mlsanity.com/paper/2202.08479/) (2022)
- [Preregistering NLP Research](https://mlsanity.com/paper/2103.06944/) (2021)
- [Using Prior Knowledge to Guide BERT's Attention in Semantic Textual Matching Tasks](https://mlsanity.com/paper/2102.10934/) (2021)
- [Identifying Machine-Paraphrased Plagiarism](https://mlsanity.com/paper/2103.11909/) (2021)
- [Neural semi-Markov CRF for Monolingual Word Alignment](https://mlsanity.com/paper/2106.02569/) (2021)
- [LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short Text Matching](https://mlsanity.com/paper/2102.12671/) (2021)
- [Dual-View Distilled BERT for Sentence Embedding](https://mlsanity.com/paper/2104.08675/) (2021)
- [Zero-shot Sequence Labeling for Transformer-based Sentence Classifiers](https://mlsanity.com/paper/2103.14465/) (2021)
- [Counterfactual Maximum Likelihood Estimation for Training Deep Networks](https://mlsanity.com/paper/2106.03831/) (2021)
- [Transfer learning for conflict and duplicate detection in software requirement pairs](https://mlsanity.com/paper/2301.03709/) (2023)
- [Revealing the Blind Spot of Sentence Encoder Evaluation by HEROS](https://mlsanity.com/paper/2306.05083/) (2023)
- [Context-Aware Interaction Network for Question Matching](https://mlsanity.com/paper/2104.08451/) (2021)
- [Attention-based model for predicting question relatedness on Stack Overflow](https://mlsanity.com/paper/2103.10763/) (2021)
- [BERT-DRE: BERT with Deep Recursive Encoder for Natural Language Sentence Matching](https://mlsanity.com/paper/2111.02188/) (2021)
- [Word Alignment in the Era of Deep Learning: A Tutorial](https://mlsanity.com/paper/2212.00138/) (2022)
- [LadRa-Net: Locally-Aware Dynamic Re-read Attention Net for Sentence Semantic Matching](https://mlsanity.com/paper/2108.02915/) (2021)
- [Description-Enhanced Label Embedding Contrastive Learning for Text Classification](https://mlsanity.com/paper/2306.08817/) (2023)
- [Zero-shot Code-Mixed Offensive Span Identification through Rationale Extraction](https://mlsanity.com/paper/2205.06119/) (2022)
- [DGA-Net Dynamic Gaussian Attention Network for Sentence Semantic Matching](https://mlsanity.com/paper/2106.04905/) (2021)
- [Contextualized Embeddings based Convolutional Neural Networks for Duplicate Question Identification](https://mlsanity.com/paper/2109.01560/) (2021)
- [An Inference Approach To Question Answering Over Knowledge Graphs](https://mlsanity.com/paper/2112.11070/) (2021)
- [GAPX: Generalized Autoregressive Paraphrase-Identification X](https://mlsanity.com/paper/2210.01979/) (2022)
- [Overlap, Unique and Conflict: Can LLMs Extract What They Can Recognize?](https://mlsanity.com/paper/2609.38799/) (2026)

## Abstract (arXiv)

In this paper, we analyze several neural network designs (and their variations) for sentence pair modeling and compare their performance extensively across eight datasets, including paraphrase identification, semantic textual similarity, natural language inference, and question answering tasks. Although most of these models have claimed state-of-the-art performance, the original papers often reported on only one or two selected datasets. We provide a systematic study and show that (i) encoding contextual information by LSTM and inter-sentence interactions are critical, (ii) Tree-LSTM does not help as much as previously claimed but surprisingly improves performance on Twitter datasets, (iii) the Enhanced Sequential Inference Model is the best so far for larger datasets, while the Pairwise Word Interaction Model achieves the best performance when less data is available. We release our implementations as an open-source toolkit.