# Paraphrase Detection: Human vs. Machine Content

arXiv: https://arxiv.org/abs/2303.13989 · HTML: https://mlsanity.com/paper/2303.13989/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 54 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [PAWS: Paraphrase Adversaries from Word Scrambling](https://mlsanity.com/paper/1904.01130/) (2019)
- [Bilateral Multi-Perspective Matching for Natural Language Sentences](https://mlsanity.com/paper/1702.03814/) (2017)
- [Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://mlsanity.com/paper/2004.14564/) (2020)
- [Aspect Sentiment Quad Prediction as Paraphrase Generation](https://mlsanity.com/paper/2110.00796/) (2021)
- [Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text](https://mlsanity.com/paper/2107.01294/) (2021)
- [A Continuously Growing Dataset of Sentential Paraphrases](https://mlsanity.com/paper/1708.00391/) (2017)
- [Controllable Text Simplification with Explicit Paraphrasing](https://mlsanity.com/paper/2010.11004/) (2020)
- [Learning to Paraphrase for Question Answering](https://mlsanity.com/paper/1708.06022/) (2017)
- [ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations](https://mlsanity.com/paper/1711.05732/) (2017)
- [How Large Language Models are Transforming Machine-Paraphrased Plagiarism](https://mlsanity.com/paper/2210.03568/) (2022)
- [ParaSCI: A Large Scientific Paraphrase Dataset for Longer Paraphrase Generation](https://mlsanity.com/paper/2101.08382/) (2021)
- [Paraphrase Generation with Latent Bag of Words](https://mlsanity.com/paper/2001.01941/) (2020)
- [Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing](https://mlsanity.com/paper/2005.13485/) (2020)
- [Improving Paraphrase Detection with the Adversarial Paraphrasing Task](https://mlsanity.com/paper/2106.07691/) (2021)
- [Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection](https://mlsanity.com/paper/2103.12450/) (2021)
- [Identifying Machine-Paraphrased Plagiarism](https://mlsanity.com/paper/2103.11909/) (2021)
- [AI Usage Cards: Responsibly Reporting AI-generated Content](https://mlsanity.com/paper/2303.03886/) (2023)
- [ConRPG: Paraphrase Generation using Contexts as Regularizer](https://mlsanity.com/paper/2109.00363/) (2021)
- [Small but Mighty: New Benchmarks for Split and Rephrase](https://mlsanity.com/paper/2009.08560/) (2020)
- [Enhanced word embeddings using multi-semantic representation through lexical chains](https://mlsanity.com/paper/2101.09023/) (2021)
- [Incorporating Word Sense Disambiguation in Neural Language Models](https://mlsanity.com/paper/2106.07967/) (2021)

## Built on by (most cited first, 12 of 12)

- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [A Survey on Detection of LLMs-Generated Content](https://mlsanity.com/paper/2310.15654/) (2023)
- [Multi-Agent Large Language Models for Conversational Task-Solving](https://mlsanity.com/paper/2410.22932/) (2024)
- [A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts](https://mlsanity.com/paper/2311.08374/) (2023)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [G3Detector: General GPT-Generated Text Detector](https://mlsanity.com/paper/2305.12680/) (2023)
- [Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection](https://mlsanity.com/paper/2402.00412/) (2024)
- [What's Wrong? Refining Meeting Summaries with LLM Feedback](https://mlsanity.com/paper/2407.11919/) (2024)
- [Paraphrase Types for Generation and Detection](https://mlsanity.com/paper/2310.14863/) (2023)
- [PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection](https://mlsanity.com/paper/2406.16288/) (2024)
- [Loci Similes: A Benchmark for Extracting Intertextualities in Latin Literature](https://mlsanity.com/paper/2601.07533/) (2026)
- [Unraveling Interwoven Roles of Large Language Models in Authorship Privacy: Obfuscation, Mimicking, and Verification](https://mlsanity.com/paper/2505.14195/) (2025)

## Abstract (arXiv)

The growing prominence of large language models, such as GPT-4 and ChatGPT, has led to increased concerns over academic integrity due to the potential for machine-generated content and paraphrasing. Although studies have explored the detection of human- and machine-paraphrased content, the comparison between these types of content remains underexplored. In this paper, we conduct a comprehensive analysis of various datasets commonly employed for paraphrase detection tasks and evaluate an array of detection methods. Our findings highlight the strengths and limitations of different detection methods in terms of performance on individual datasets, revealing a lack of suitable machine-generated datasets that can be aligned with human expectations. Our main finding is that human-authored paraphrases exceed machine-generated ones in terms of difficulty, diversity, and similarity implying that automatically generated texts are not yet on par with human-level performance. Transformers emerged as the most effective method across datasets with TF-IDF excelling on semantically diverse corpora. Additionally, we identify four datasets as the most diverse and challenging for paraphrase detection.