mlsanityData to 7 Oct 2026

DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

arXiv 2301.11305 ↗First posted 26 Jan 2023cs.CL cs.AIcited by 705

Not on the map. Why?

Cited by 111 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The increasing fluency and widespread usage of large language models (LLMs) highlight the desirability of corresponding tools aiding detection of LLM-generated text. In this paper, we identify a property of the structure of an LLM's probability function that is useful for such detection. Specifically, we demonstrate that text sampled from an LLM tends to occupy negative curvature regions of the model's log probability function. Leveraging this observation, we then define a new curvature-based criterion for judging if a passage is generated from a given LLM. This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g., T5). We find DetectGPT is more discriminative than existing zero-shot methods for model sample detection, notably improving detection of fake news articles generated by 20B parameter GPT-NeoX from 0.81 AUROC for the strongest zero-shot baseline to 0.95 AUROC for DetectGPT. See https://ericmitchell.ai/detectgpt for code, data, and other project information.

Key facts. Cited by 705 papers in the map (data through 7 Oct 2026). Builds on 26 of its 47 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

26 of 47 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  12. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
14 more
  1. 2018Hierarchical Neural Story Generationcited by 1,182
  2. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  3. 2023A Watermark for Large Language Modelscited by 667
  4. 2019Defending Against Neural Fake Newscited by 553
  5. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  6. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
  7. 2021Multi-attentional Deepfake Detectioncited by 339
  8. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 260
  9. 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 207
  10. 2020Automatic Detection of Machine Generated Text: A Critical Surveycited by 160
  11. 2020The DeepFake Detection Challenge (DFDC) Datasetcited by 125
  12. 2020TweepFake: about Detecting Deepfake Tweetscited by 117
  13. 2019Real or Fake? Learning to Discriminate Machine from Human Generated Textcited by 98
  14. 2020DeepFake Detection by Analyzing Convolutional Tracescited by 81

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 142; 2024: 179; 2025: 185; 2026: 199.

705 papers.

Sort:

By area:

  1. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  2. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
  3. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  4. 2023Can AI-Generated Text be Reliably Detected?cited by 372
  5. 2023Robust Distortion-free Watermarks for Language Modelscited by 279
  6. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 278
  7. 2023Provable Robust Watermarking for AI-Generated Textcited by 259
  8. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 255
  9. 2023Challenges and Applications of Large Language Modelscited by 251
  10. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  11. 2023Self-Consuming Generative Models Go MADcited by 239
  12. 2023Undetectable Watermarks for Language Modelscited by 208
38 more
  1. 2023DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Textcited by 207
  2. 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 187
  3. 2023GPT detectors are biased against non-native English writerscited by 175
  4. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  5. 2023On the Reliability of Watermarks for Large Language Modelscited by 170
  6. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
  7. 2024RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectorscited by 157
  8. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 136
  9. 2023MAGE: Machine-generated Text Detection in the Wildcited by 130
  10. 2023DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Textcited by 127
  11. 2023The Science of Detecting LLM-Generated Textscited by 126
  12. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  13. 2023M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detectioncited by 118
  14. 2023Who Wrote this Code? Watermarking for Code Generationcited by 116
  15. 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 112
  16. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 111
  17. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 109
  18. 2024LLM Dataset Inference: Did you train on my dataset?cited by 107
  19. 2023Regulating ChatGPT and other Large Generative AI Modelscited by 102
  20. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 102
  21. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  22. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
  23. 2023Contextual Object Detection with Multimodal Large Language Modelscited by 100
  24. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 98
  25. 2023MGTBench: Benchmarking Machine-Generated Text Detectioncited by 90
  26. 2023Ghostbuster: Detecting Text Ghostwritten by Large Language Modelscited by 90
  27. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 90
  28. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 90
  29. 2023On the Possibilities of AI-Generated Text Detectioncited by 88
  30. 2023OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examplescited by 80
  31. 2023Three Bricks to Consolidate Watermarks for Large Language Modelscited by 80
  32. 2024Watermark Stealing in Large Language Modelscited by 79
  33. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Modelscited by 77
  34. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  35. 2023Evading Watermark based Detection of AI-Generated Contentcited by 74
  36. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73
  37. 2023On the Learnability of Watermarks for Language Modelscited by 72
  38. 2023A Survey on Detection of LLMs-Generated Contentcited by 70

50 most cited of 705 All 705 (JSON)