mlsanityData to 7 Oct 2026

Can AI-Generated Text be Reliably Detected?

arXiv 2303.11156 ↗First posted 17 Mar 2023cs.CL cs.AI cs.LGcited by 372

On the map: LLM safety and security → Watermarking generated content and models

Abstract (arXiv)

Large Language Models (LLMs) perform impressively well in various applications. However, the potential for misuse of these models in activities such as plagiarism, generating fake news, and spamming has raised concern about their responsible use. Consequently, the reliable detection of AI-generated text has become a critical area of research. AI text detectors have shown to be effective under their specific settings. In this paper, we stress-test the robustness of these AI text detectors in the presence of an attacker. We introduce recursive paraphrasing attack to stress test a wide range of detection schemes, including the ones using the watermarking as well as neural network-based detectors, zero shot classifiers, and retrieval-based detectors. Our experiments conducted on passages, each approximately 300 tokens long, reveal the varying sensitivities of these detectors to our attacks. Our findings indicate that while our recursive paraphrasing method can significantly reduce detection rates, it only slightly degrades text quality in many cases, highlighting potential vulnerabilities in current detection systems in the presence of an attacker. Additionally, we investigate the susceptibility of watermarked LLMs to spoofing attacks aimed at misclassifying human-written text as AI-generated. We demonstrate that an attacker can infer hidden AI text signatures without white-box access to the detection method, potentially leading to reputational risks for LLM developers. Finally, we provide a theoretical framework connecting the AUROC of the best possible detector to the Total Variation distance between human and AI text distributions. This analysis offers insights into the fundamental challenges of reliable detection as language models continue to advance. Our code is publicly available at https://github.com/vinusankars/Reliability-of-AI-text-detectors.

Key facts. Cited by 372 papers in the map (data through 7 Oct 2026). Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security. Builds on 29 of its 52 references that are in the map. First posted on arXiv 17 Mar 2023.

Builds on

29 of 52 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Protecting Language Generation Models via Invisible Watermarking6 Feb
  2. 2023A Watermark for Large Language Models24 Jan

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  11. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
15 more
  1. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  2. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  3. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
  4. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 260
  5. 2020Automatic Detection of Machine Generated Text: A Critical Surveycited by 160
  6. 2020TweepFake: about Detecting Deepfake Tweetscited by 117
  7. 2019On the Use of ArXiv as a Datasetcited by 101
  8. 2019Real or Fake? Learning to Discriminate Machine from Human Generated Textcited by 98
  9. 2019Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detectioncited by 50
  10. 2022Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregationcited by 50
  11. 2020Attacking Neural Text Detectorscited by 37
  12. 2022Deepfake Text Detection: Limitations and Opportunitiescited by 37
  13. 2023CUDA: Convolution-based Unlearnable Datasetscited by 26
  14. 2022Certifying Model Accuracy under Distribution Shiftscited by 5
  15. 2016The Quality of the Covariance Selection Through Detection Problem and AUC Boundscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 94; 2024: 120; 2025: 80; 2026: 78.

372 papers.

Sort:

By area:

  1. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
  2. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 278
  3. 2023Provable Robust Watermarking for AI-Generated Textcited by 259
  4. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 255
  5. 2023Challenges and Applications of Large Language Modelscited by 251
  6. 2023Undetectable Watermarks for Language Modelscited by 208
  7. 2023DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Textcited by 207
  8. 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 187
  9. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  10. 2023GPT detectors are biased against non-native English writerscited by 175
  11. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  12. 2023On the Reliability of Watermarks for Large Language Modelscited by 170
38 more
  1. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
  2. 2024RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectorscited by 157
  3. 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
  4. 2023MAGE: Machine-generated Text Detection in the Wildcited by 130
  5. 2023DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Textcited by 127
  6. 2023Factuality Challenges in the Era of Large Language Modelscited by 125
  7. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  8. 2023M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detectioncited by 118
  9. 2023Who Wrote this Code? Watermarking for Code Generationcited by 116
  10. 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 112
  11. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 111
  12. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 102
  13. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 102
  14. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
  15. 2023Contextual Object Detection with Multimodal Large Language Modelscited by 100
  16. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 98
  17. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 97
  18. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
  19. 2023Ghostbuster: Detecting Text Ghostwritten by Large Language Modelscited by 90
  20. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 90
  21. 2023On the Possibilities of AI-Generated Text Detectioncited by 88
  22. 2023OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examplescited by 80
  23. 2024Watermark Stealing in Large Language Modelscited by 79
  24. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Modelscited by 77
  25. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  26. 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 75
  27. 2023On the Learnability of Watermarks for Language Modelscited by 72
  28. 2023A Survey on Detection of LLMs-Generated Contentcited by 70
  29. 2023Publicly-Detectable Watermarking for Language Modelscited by 69
  30. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  31. 2023Contrasting Linguistic Patterns in Human and LLM-Generated News Textcited by 62
  32. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarkscited by 62
  33. 2024Adaptive Text Watermark for Large Language Modelscited by 60
  34. 2023Red Teaming Language Model Detectors with Language Modelscited by 55
  35. 2023An Unforgeable Publicly Verifiable Watermark for Large Language Modelscited by 54
  36. 2023Fighting Fire with Fire: The Dual Role of LLMs in Crafting and Detecting Elusive Disinformationcited by 51
  37. 2024Raidar: geneRative AI Detection viA Rewritingcited by 51
  38. 2024Community Forensics: Using Thousands of Generators to Train Fake Image Detectorscited by 50

50 most cited of 372 All 372 (JSON)