# Neural Paraphrase Identification of Questions with Noisy Pretraining

arXiv: https://arxiv.org/abs/1704.04565 · HTML: https://mlsanity.com/paper/1704.04565/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2017.

## Built on by (most cited first, 9 of 9)

- [Using Prior Knowledge to Guide BERT's Attention in Semantic Textual Matching Tasks](https://mlsanity.com/paper/2102.10934/) (2021)
- [Paraphrase Identification with Deep Learning: A Review of Datasets and Methods](https://mlsanity.com/paper/2212.06933/) (2022)
- [BERT-DRE: BERT with Deep Recursive Encoder for Natural Language Sentence Matching](https://mlsanity.com/paper/2111.02188/) (2021)
- [Predicting the Success of Domain Adaptation in Text Similarity](https://mlsanity.com/paper/2106.04641/) (2021)
- [Contextualized Embeddings based Convolutional Neural Networks for Duplicate Question Identification](https://mlsanity.com/paper/2109.01560/) (2021)
- [GAPX: Generalized Autoregressive Paraphrase-Identification X](https://mlsanity.com/paper/2210.01979/) (2022)
- [Using Graph Algorithms to Pretrain Graph Completion Transformers](https://mlsanity.com/paper/2210.07453/) (2022)
- [A General and Flexible Multi-concept Parsing Framework for Multilingual Semantic Matching](https://mlsanity.com/paper/2403.02975/) (2024)
- [R-DEIM Net: An Efficient Rationale-Augmented Dual-Expert Interaction Model for Paraphrase Detection](https://mlsanity.com/paper/2609.30100/) (2026)

## Abstract (arXiv)

We present a solution to the problem of paraphrase identification of questions. We focus on a recent dataset of question pairs annotated with binary paraphrase labels and show that a variant of the decomposable attention model (Parikh et al., 2016) results in accurate performance on this task, while being far simpler than many competing neural architectures. Furthermore, when the model is pretrained on a noisy dataset of automatically collected question paraphrases, it obtains the best reported performance on the dataset.