# Contextualized Embeddings based Convolutional Neural Networks for Duplicate Question Identification

arXiv: https://arxiv.org/abs/2109.01560 · HTML: https://mlsanity.com/paper/2109.01560/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 3 Sep 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Enhanced LSTM for Natural Language Inference](https://mlsanity.com/paper/1609.06038/) (2016)
- [Bilateral Multi-Perspective Matching for Natural Language Sentences](https://mlsanity.com/paper/1702.03814/) (2017)
- [Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning](https://mlsanity.com/paper/1804.00079/) (2018)
- [A Fast Unified Model for Parsing and Sentence Understanding](https://mlsanity.com/paper/1603.06021/) (2016)
- [Natural Language Inference over Interaction Space](https://mlsanity.com/paper/1709.04348/) (2017)
- [A Compare-Aggregate Model for Matching Text Sequences](https://mlsanity.com/paper/1611.01747/) (2016)
- [Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering](https://mlsanity.com/paper/1806.04330/) (2018)
- [Deep Learning for Answer Sentence Selection](https://mlsanity.com/paper/1412.1632/) (2014)
- [Sentence Similarity Learning by Lexical Decomposition and Composition](https://mlsanity.com/paper/1602.07019/) (2016)
- [Neural Paraphrase Identification of Questions with Noisy Pretraining](https://mlsanity.com/paper/1704.04565/) (2017)
- [Evolution of transfer learning in natural language processing](https://mlsanity.com/paper/1910.07370/) (2019)
- [Shortcut-Stacked Sentence Encoders for Multi-Domain Inference](https://mlsanity.com/paper/1708.02312/) (2017)
- [REGMAPR - Text Matching Made Easy](https://mlsanity.com/paper/1808.04343/) (2018)
- [Cell-aware Stacked LSTMs for Modeling Sentences](https://mlsanity.com/paper/1809.02279/) (2018)
- [Applying Transfer Learning for Improving Domain-Specific Search Experience Using Query to Question Similarity](https://mlsanity.com/paper/2101.02351/) (2021)
- [Siamese Neural Networks with Random Forest for detecting duplicate question pairs](https://mlsanity.com/paper/1801.07288/) (2018)

## Abstract (arXiv)

Question Paraphrase Identification (QPI) is a critical task for large-scale Question-Answering forums. The purpose of QPI is to determine whether a given pair of questions are semantically identical or not. Previous approaches for this task have yielded promising results, but have often relied on complex recurrence mechanisms that are expensive and time-consuming in nature. In this paper, we propose a novel architecture combining a Bidirectional Transformer Encoder with Convolutional Neural Networks for the QPI task. We produce the predictions from the proposed architecture using two different inference setups: Siamese and Matched Aggregation. Experimental results demonstrate that our model achieves state-of-the-art performance on the Quora Question Pairs dataset. We empirically prove that the addition of convolution layers to the model architecture improves the results in both inference setups. We also investigate the impact of partial and complete fine-tuning and analyze the trade-off between computational power and accuracy in the process. Based on the obtained results, we conclude that the Matched-Aggregation setup consistently outperforms the Siamese setup. Our work provides insights into what architecture combinations and setups are likely to produce better results for the QPI task.