mlsanityData to 7 Oct 2026

Paraphrase Identification with Deep Learning: A Review of Datasets and Methods

arXiv 2212.06933 ↗First posted 13 Dec 2022cs.CL cs.AI cs.IRcited by 13

Not on the map. Why?

Abstract (arXiv)

The rapid progress of Natural Language Processing (NLP) technologies has led to the widespread availability and effectiveness of text generation tools such as ChatGPT and Claude. While highly useful, these technologies also pose significant risks to the credibility of various media forms if they are employed for paraphrased plagiarism -- one of the most subtle forms of content misuse in scientific literature and general text media. Although automated methods for paraphrase identification have been developed, detecting this type of plagiarism remains challenging due to the inconsistent nature of the datasets used to train these methods. In this article, we examine traditional and contemporary approaches to paraphrase identification, investigating how the under-representation of certain paraphrase types in popular datasets, including those used to train Large Language Models (LLMs), affects the ability to detect plagiarism. We introduce and validate a new refined typology for paraphrases (ReParaphrased, REfined PARAPHRASE typology definitions) to better understand the disparities in paraphrase type representation. Lastly, we propose new directions for future research and dataset development to enhance AI-based paraphrase detection.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 73 of its 226 references that are in the map. First posted on arXiv 13 Dec 2022.

Builds on

73 of 226 references.

References · 73

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  8. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  11. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  12. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
38 more
  1. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  2. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  3. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  4. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  5. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  6. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  7. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  8. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  9. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  10. 2014Memory Networkscited by 354
  11. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  12. 2015Skip-Thought Vectorscited by 321
  13. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 305
  14. 2016A Decomposable Attention Model for Natural Language Inferencecited by 214
  15. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 205
  16. 2016Enhanced LSTM for Natural Language Inferencecited by 199
  17. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  18. 2014A Convolutional Neural Network for Modelling Sentencescited by 163
  19. 2016Long Short-Term Memory-Networks for Machine Readingcited by 161
  20. 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 158
  21. 2020Evaluating the Evaluation of Diversity in Natural Language Generationcited by 127
  22. 2015Neural Responding Machine for Short-Text Conversationcited by 124
  23. 2017Bilateral Multi-Perspective Matching for Natural Language Sentencescited by 120
  24. 2019StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understandingcited by 108
  25. 2019Dynamic Curriculum Learning for Imbalanced Data Classificationcited by 93
  26. 2015Convolutional Neural Network Architectures for Matching Natural Language Sentencescited by 82
  27. 2020Regularizing Neural Networks via Adversarial Model Perturbationcited by 78
  28. 2017DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understandingcited by 71
  29. 2015Reasoning about Entailment with Neural Attentioncited by 68
  30. 2016A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Taskscited by 62
  31. 2020RealFormer: Transformer Likes Residual Attentioncited by 62
  32. 2016A Simple, Fast Diverse Decoding Algorithm for Neural Generationcited by 58
  33. 2017Paraphrase Generation with Deep Reinforcement Learningcited by 58
  34. 2017Neural AMR: Sequence-to-Sequence Models for Parsing and Generationcited by 56
  35. 2018Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanismscited by 53
  36. 2016Neural Paraphrase Generation with Stacked Residual LSTM Networkscited by 52
  37. 2021Structural Adapters in Pretrained Language Models for AMR-to-text Generationcited by 48
  38. 2020Adversarial Semantic Collisionscited by 42

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 4; 2026: 3.

13 papers.

Sort:

By area:

  1. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 152
  2. 2023One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Eracited by 55
  3. 2025MedArabiQ: Benchmarking Large Language Models on Arabic Medical Taskscited by 15
  4. 2024Towards Human Understanding of Paraphrase Types in Large Language Modelscited by 2
  5. 2024BERT-Enhanced Retrieval Tool for Homework Plagiarism Detection Systemcited by 1
  6. 2024What's Mine becomes Yours: Defining, Annotating and Detecting Context-Dependent Paraphrases in News Interview Dialogscited by 1
  7. 2025Attacks and Defenses Against LLM Fingerprintingcited by 1
  8. 2026DoGMaTiQ: Automated Generation of Question-and-Answer Nuggets for Report Evaluationcited by 1
  9. 2023Word Definitions from Large Language Modelscited by 0
  10. 2025Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Datacited by 0
  11. 2025Semantic Reconstruction of Adversarial Plagiarism: A Context-Aware Framework for Detecting and Restoring "Tortured Phrases" in Scientific Literaturecited by 0
  12. 2026From transcription to semantic corpus analysis: unsupervised learning of sentence representations for ancient languagescited by 0
1 more
  1. 2026R-DEIM Net: An Efficient Rationale-Augmented Dual-Expert Interaction Model for Paraphrase Detectioncited by 0