# Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection

arXiv: https://arxiv.org/abs/2103.12450 · HTML: https://mlsanity.com/paper/2103.12450/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Identifying Machine-Paraphrased Plagiarism](https://mlsanity.com/paper/2103.11909/) (2021)

## Built on by (most cited first, 17 of 17)

- [Voting or Consensus? Decision-Making in Multi-Agent Debate](https://mlsanity.com/paper/2502.19130/) (2025)
- [How Large Language Models are Transforming Machine-Paraphrased Plagiarism](https://mlsanity.com/paper/2210.03568/) (2022)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [A Domain-adaptive Pre-training Approach for Language Bias Detection in News](https://mlsanity.com/paper/2205.10773/) (2022)
- [Paraphrase Identification with Deep Learning: A Review of Datasets and Methods](https://mlsanity.com/paper/2212.06933/) (2022)
- [Specialized Document Embeddings for Aspect-based Similarity of Research Papers](https://mlsanity.com/paper/2203.14541/) (2022)
- [Paraphrase Detection: Human vs. Machine Content](https://mlsanity.com/paper/2303.13989/) (2023)
- [Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detection](https://mlsanity.com/paper/2111.07819/) (2021)
- [Incorporating Word Sense Disambiguation in Neural Language Models](https://mlsanity.com/paper/2106.07967/) (2021)
- [Paraphrase Types for Generation and Detection](https://mlsanity.com/paper/2310.14863/) (2023)
- [PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection](https://mlsanity.com/paper/2406.16288/) (2024)
- [Overview of the Plagiarism Detection Task at PAN 2025](https://mlsanity.com/paper/2510.06805/) (2025)
- [L-PRISMA: An Extension of PRISMA in the Era of Generative Artificial Intelligence (GenAI)](https://mlsanity.com/paper/2603.19236/) (2026)
- [PerPaDa: A Persian Paraphrase Dataset based on Implicit Crowdsourcing Data Collection](https://mlsanity.com/paper/2201.06573/) (2022)
- [MuLVE, A Multi-Language Vocabulary Evaluation Data Set](https://mlsanity.com/paper/2201.06286/) (2022)
- [Generation, Evaluation, and Explanation of Novelists' Styles with Single-Token Prompts](https://mlsanity.com/paper/2511.20459/) (2025)
- [Beyond Representational Similarity: Source-Conditioned Description-Length Gain for Generative Plagiarism Detection and Candidate Source Reranking](https://mlsanity.com/paper/2608.03859/) (2026)

## Abstract (arXiv)

The rise of language models such as BERT allows for high-quality text paraphrasing. This is a problem to academic integrity, as it is difficult to differentiate between original and machine-generated content. We propose a benchmark consisting of paraphrased articles using recent language models relying on the Transformer architecture. Our contribution fosters future research of paraphrase detection systems as it offers a large collection of aligned original and paraphrased documents, a study regarding its structure, classification experiments with state-of-the-art systems, and we make our findings publicly available.