# Corpus-Based Paraphrase Detection Experiments and Review

arXiv: https://arxiv.org/abs/2106.00145 · HTML: https://mlsanity.com/paper/2106.00145/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 74 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [A Survey Of Cross-lingual Word Embedding Models](https://mlsanity.com/paper/1706.04902/) (2017)
- [Paraphrase Generation with Deep Reinforcement Learning](https://mlsanity.com/paper/1711.00279/) (2017)
- [Testing of Support Tools for Plagiarism Detection](https://mlsanity.com/paper/2002.04279/) (2020)
- [A Deep Network Model for Paraphrase Detection in Short Text Messages](https://mlsanity.com/paper/1712.02820/) (2017)
- [Evaluating Neural Word Representations in Tensor-Based Compositional Settings](https://mlsanity.com/paper/1408.6179/) (2014)
- [Learning Multi-Modal Word Representation Grounded in Visual Context](https://mlsanity.com/paper/1711.03483/) (2017)
- [Improving Tree-LSTM with Tree Attention](https://mlsanity.com/paper/1901.00066/) (2019)
- [Syntax-Aware Multi-Sense Word Embeddings for Deep Compositional Models of Meaning](https://mlsanity.com/paper/1508.02354/) (2015)
- [Methods for Detecting Paraphrase Plagiarism](https://mlsanity.com/paper/1712.10309/) (2017)

## Built on by (most cited first, 5 of 5)

- [Paraphrase Identification with Deep Learning: A Review of Datasets and Methods](https://mlsanity.com/paper/2212.06933/) (2022)
- [A Bayesian approach for prompt optimization in pre-trained language models](https://mlsanity.com/paper/2312.00471/) (2023)
- [Semantic Search as Extractive Paraphrase Span Detection](https://mlsanity.com/paper/2112.04886/) (2021)
- [Discourse Features Enhance Detection of Document-Level Machine-Generated Content](https://mlsanity.com/paper/2412.12679/) (2024)
- [The power of text similarity in identifying AI-LLM paraphrased documents: The case of BBC news articles and ChatGPT](https://mlsanity.com/paper/2505.12405/) (2025)

## Abstract (arXiv)

Paraphrase detection is important for a number of applications, including plagiarism detection, authorship attribution, question answering, text summarization, text mining in general, etc. In this paper, we give a performance overview of various types of corpus-based models, especially deep learning (DL) models, with the task of paraphrase detection. We report the results of eight models (LSI, TF-IDF, Word2Vec, Doc2Vec, GloVe, FastText, ELMO, and USE) evaluated on three different public available corpora: Microsoft Research Paraphrase Corpus, Clough and Stevenson and Webis Crowd Paraphrase Corpus 2011. Through a great number of experiments, we decided on the most appropriate approaches for text pre-processing: hyper-parameters, sub-model selection-where they exist (e.g., Skipgram vs. CBOW), distance measures, and semantic similarity/paraphrase detection threshold. Our findings and those of other researchers who have used deep learning models show that DL models are very competitive with traditional state-of-the-art approaches and have potential that should be further developed.