mlsanityData to 7 Oct 2026

Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

arXiv 2303.13408 ↗First posted 23 Mar 2023cs.CL cs.CR cs.LGcited by 401

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

The rise in malicious usage of large language models, such as fake content creation and academic plagiarism, has motivated the development of approaches that identify AI-generated text, including those based on watermarking or outlier detection. However, the robustness of these detection algorithms to paraphrases of AI-generated text remains unclear. To stress test these detectors, we build a 11B parameter paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition on surrounding context, and control lexical diversity and content reordering. Using DIPPER to paraphrase text generated by three large language models (including GPT3.5-davinci-003) successfully evades several detectors, including watermarking, GPTZero, DetectGPT, and OpenAI's text classifier. For example, DIPPER drops detection accuracy of DetectGPT from 70.3% to 4.6% (at a constant false positive rate of 1%), without appreciably modifying the input semantics. To increase the robustness of AI-generated text detection to paraphrase attacks, we introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider. Given a candidate text, our algorithm searches a database of sequences previously generated by the API, looking for sequences that match the candidate text within a certain threshold. We empirically verify our defense using a database of 15M generations from a fine-tuned T5-XXL model and find that it can detect 80% to 97% of paraphrased generations across different settings while only classifying 1% of human-written sequences as AI-generated. We open-source our models, code and data.

Key facts. Cited by 401 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 69 of its 125 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

69 of 125 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Can AI-Generated Text be Reliably Detected?17 Mar
  2. 2023A Watermark for Large Language Models24 Jan
  3. 2022The Ethical Need for Watermarks in Machine-Generated Language7 Sep

Other references · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2016Pointer Sentinel Mixture Modelscited by 2,992
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  11. 2017Billion-scale similarity search with GPUscited by 2,477
  12. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
38 more
  1. 2020Extracting Training Data from Large Language Modelscited by 2,144
  2. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  3. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  4. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  5. 2019ELI5: Long Form Question Answeringcited by 567
  6. 2019Defending Against Neural Fake Newscited by 553
  7. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
  8. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  9. 2020Evaluation of Text Generation: A Surveycited by 275
  10. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 260
  11. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  12. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 181
  13. 2020Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hidingcited by 121
  14. 2019Real or Fake? Learning to Discriminate Machine from Human Generated Textcited by 98
  15. 2018Context-Aware Neural Machine Translation Learns Anaphora Resolutioncited by 97
  16. 2017Neural Machine Translation with Extended Contextcited by 95
  17. 2020Residual Energy-Based Models for Text Generationcited by 95
  18. 2020Pre-training via Paraphrasingcited by 92
  19. 2019Do Massively Pretrained Language Models Make Better Storytellers?cited by 90
  20. 2018Document-Level Neural Machine Translation with Hierarchical Attention Networkscited by 85
  21. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  22. 2018Improving the Transformer Translation Model with Document-Level Contextcited by 74
  23. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  24. 2017Paraphrase Generation with Deep Reinforcement Learningcited by 58
  25. 2019Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translationcited by 56
  26. 2020Neural Syntactic Preordering for Controlled Paraphrase Generationcited by 56
  27. 2021Paragraph-level Simplification of Medical Textscited by 52
  28. 2017Exploiting Cross-Sentence Context for Neural Machine Translationcited by 48
  29. 2020Syntax-guided Controlled Generation of Paraphrasescited by 47
  30. 2021Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairscited by 45
  31. 2021Keep it Simple: Unsupervised Simplification of Multi-Paragraph Textcited by 45
  32. 2022Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literaturecited by 45
  33. 2019Controllable Paraphrase Generation with a Syntactic Exemplarcited by 44
  34. 2019Context-Aware Monolingual Repair for Neural Machine Translationcited by 41
  35. 2017Does Neural Machine Translation Benefit from Larger Context?cited by 38
  36. 2019ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translationcited by 38
  37. 2021When Does Translation Require Context? A Data-driven, Multilingual Explorationcited by 36
  38. 2019Paraphrasing with Large Language Modelscited by 35

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 70; 2024: 106; 2025: 120; 2026: 105.

401 papers.

Sort:

By area:

  1. 2023Provable Robust Watermarking for AI-Generated Textcited by 259
  2. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 255
  3. 2023Challenges and Applications of Large Language Modelscited by 251
  4. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  5. 2023Undetectable Watermarks for Language Modelscited by 208
  6. 2023DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Textcited by 207
  7. 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 187
  8. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  9. 2023GPT detectors are biased against non-native English writerscited by 175
  10. 2023On the Reliability of Watermarks for Large Language Modelscited by 170
  11. 2024RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectorscited by 157
  12. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
38 more
  1. 2023MAGE: Machine-generated Text Detection in the Wildcited by 130
  2. 2023DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Textcited by 127
  3. 2023Who Wrote this Code? Watermarking for Code Generationcited by 116
  4. 2024Attacking Vision-Language Computer Agents via Pop-upscited by 112
  5. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 111
  6. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 102
  7. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
  8. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 97
  9. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 90
  10. 2023On the Possibilities of AI-Generated Text Detectioncited by 88
  11. 2023Testing of Detection Tools for AI-Generated Textcited by 86
  12. 2023OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examplescited by 80
  13. 2024MarkLLM: An Open-Source Toolkit for LLM Watermarkingcited by 80
  14. 2024Watermark Stealing in Large Language Modelscited by 79
  15. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Modelscited by 77
  16. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  17. 2023Evading Watermark based Detection of AI-Generated Contentcited by 74
  18. 2023On the Learnability of Watermarks for Language Modelscited by 72
  19. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 72
  20. 2023A Survey on Detection of LLMs-Generated Contentcited by 70
  21. 2023Publicly-Detectable Watermarking for Language Modelscited by 69
  22. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarkscited by 62
  23. 2024Adaptive Text Watermark for Large Language Modelscited by 60
  24. 2023Red Teaming Language Model Detectors with Language Modelscited by 55
  25. 2024DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learningcited by 52
  26. 2023Advancing Beyond Identification: Multi-bit Watermark for Large Language Modelscited by 51
  27. 2023Fighting Fire with Fire: The Dual Role of LLMs in Crafting and Detecting Elusive Disinformationcited by 51
  28. 2024Raidar: geneRative AI Detection viA Rewritingcited by 51
  29. 2024Community Forensics: Using Thousands of Generators to Train Fake Image Detectorscited by 50
  30. 2023A Resilient and Accessible Distribution-Preserving Watermark for Large Language Modelscited by 47
  31. 2024No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choicescited by 47
  32. 2024k-SemStamp: A Clustering-Based Semantic Watermark for Detection of Machine-Generated Textcited by 46
  33. 2023Large Language Models can be Guided to Evade AI-Generated Text Detectioncited by 43
  34. 2023Towards Codable Watermarking for Injecting Multi-bits Information to LLMscited by 43
  35. 2024M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detectioncited by 43
  36. 2024SoK: Watermarking for AI-Generated Contentcited by 43
  37. 2024SUGARCREPE++ Dataset: Vision-Language Model Sensitivity to Semantic and Lexical Alterationscited by 42
  38. 2024Few-Shot Detection of Machine-Generated Text using Style Representationscited by 40

50 most cited of 401 All 401 (JSON)