mlsanityData to 13 Oct 2026

RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors

arXiv 2405.07940 ↗First posted 13 May 2024cs.CLcited by 158

Not on the map. Why?

Abstract (arXiv)

Many commercial and open-source models claim to detect machine-generated text with extremely high accuracy (99% or more). However, very few of these detectors are evaluated on shared benchmark datasets and even when they are, the datasets used for evaluation are insufficiently challenging-lacking variations in sampling strategy, adversarial attacks, and open-source generative models. In this work we present RAID: the largest and most challenging benchmark dataset for machine-generated text detection. RAID includes over 6 million generations spanning 11 models, 8 domains, 11 adversarial attacks and 4 decoding strategies. Using RAID, we evaluate the out-of-domain and adversarial robustness of 8 open- and 4 closed-source detectors and find that current detectors are easily fooled by adversarial attacks, variations in sampling strategies, repetition penalties, and unseen generative models. We release our data along with a leaderboard to encourage future research.

Key facts. Cited by 158 papers in the map (data through 13 Oct 2026). Builds on 55 of its 95 references that are in the map. First posted on arXiv 13 May 2024.

Builds on

55 of 95 references.

References · 55

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2022Training language models to follow instructions with human feedbackcited by 15,856
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  7. 2016Enriching Word Vectors with Subword Informationcited by 1,581
  8. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 760
  9. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 757
  10. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 717
  11. 2021Deduplicating Training Data Makes Language Models Bettercited by 710
  12. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 600
38 more
  1. 2023The Falcon Series of Open Language Modelscited by 533
  2. 2019Release Strategies and the Social Impacts of Language Modelscited by 511
  3. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 492
  4. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 478
  5. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 423
  6. 2023Can AI-Generated Text be Reliably Detected?cited by 383
  7. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  8. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 344
  9. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  10. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 282
  11. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 261
  12. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 258
  13. 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 188
  14. 2023GPT detectors are biased against non-native English writerscited by 176
  15. 2022Truncation Sampling as Language Model Desmoothingcited by 121
  16. 2023M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detectioncited by 120
  17. 2021TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generationcited by 119
  18. 2023MGTBench: Benchmarking Machine-Generated Text Detectioncited by 91
  19. 2023AI model GPT-3 (dis)informs us better than humanscited by 78
  20. 2019The Limitations of Stylometry for Detecting Machine-Generated Fake Newscited by 63
  21. 2022Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Textcited by 63
  22. 2023ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishingcited by 52
  23. 2023Large Language Models can be Guided to Evade AI-Generated Text Detectioncited by 44
  24. 2023LLMDet: A Third Party Large Language Models Generated Text Detection Toolcited by 44
  25. 2024Few-Shot Detection of Machine-Generated Text using Style Representationscited by 40
  26. 2020Attacking Neural Text Detectorscited by 39
  27. 2020RoFT: A Tool for Evaluating Human Detection of Machine-Generated Textcited by 36
  28. 2023HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpuscited by 34
  29. 2023Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine-Generated Text in Multiple Domainscited by 31
  30. 2022Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russiancited by 29
  31. 2023How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Promptscited by 23
  32. 2024Authorship Obfuscation in Multilingual Machine-Generated Text Detectioncited by 19
  33. 2013New Alignment Methods for Discriminative Book Summarizationcited by 18
  34. 2023How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attackscited by 18
  35. 2023Evade ChatGPT Detectors via A Single Spacecited by 18
  36. 2023Counter Turing Test CT^2: AI-Generated Text Detection is Not as Easy as You May Think -- Introducing AI Detectability Indexcited by 14
  37. 2023Evaluating AIGC Detectors on Code Contentcited by 12
  38. 2023On the Generalization of Training-based ChatGPT Detection Methodscited by 11

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 15; 2025: 62; 2026: 81.

158 papers.

Sort:

By area:

  1. 2025DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressivelycited by 61
  2. 2025People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated textcited by 39
  3. 2025The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Modelscited by 28
  4. 2024Beemo: Benchmark of Expert-edited Machine-generated Outputscited by 26
  5. 2025RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patternscited by 24
  6. 2024Detecting AI-Generated Text: Factors Influencing Detectability with Current Methodscited by 23
  7. 2025DAMAGE: Detecting Adversarially Modified AI Generated Textcited by 20
  8. 2025Is Your Paper Being Reviewed by an LLM? Benchmarking AI Text Detection in Peer Reviewcited by 19
  9. 2025Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writingcited by 17
  10. 2025Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMscited by 17
  11. 2024Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurementcited by 16
  12. 2024MiRAGeNews: Multimodal Realistic AI-Generated News Detectioncited by 15
38 more
  1. 2024MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Textscited by 14
  2. 2025Machine-generated text detection prevents language model collapsecited by 14
  3. 2025Feature-Level Insights into Artificial Text Detection with Sparse Autoencoderscited by 14
  4. 2025HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoringcited by 14
  5. 2024DALD: Improving Logits-based Detector without Logits from Black-box LLMscited by 13
  6. 2024AI "News" Content Farms Are Easy to Make and Hard to Detect: A Case Study in Italiancited by 12
  7. 2025DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learningcited by 12
  8. 2024A Practical Examination of AI-Generated Text Detectors for Large Language Modelscited by 11
  9. 2025GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Humancited by 11
  10. 2026GPTZero: Robust Detection of LLM-Generated Textscited by 11
  11. 2025Reliably Bounding False Positives: A Zero-Shot Machine-Generated Text Detection Framework via Multiscaled Conformal Predictioncited by 10
  12. 2025DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigmcited by 10
  13. 2025AuthorMist: Evading AI Text Detectors with Reinforcement Learningcited by 9
  14. 2025Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectorscited by 9
  15. 2025DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domainscited by 9
  16. 2025textttDroid: A Resource Suite for AI-Generated Code Detectioncited by 9
  17. 2026Detecting LLM-Generated Text with Performance Guaranteescited by 9
  18. 2025OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attributioncited by 8
  19. 2025Frankentext: Stitching random text fragments into long-form narrativescited by 8
  20. 2025DetectAnyLLM: Towards Generalizable and Robust Detection of Machine-Generated Text Across Domains and Modelscited by 8
  21. 2025Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detectioncited by 8
  22. 2025GenAI Content Detection Task 3: Cross-Domain Machine-Generated Text Detection Challengecited by 7
  23. 2025Authorship Attribution in Multilingual Machine-Generated Textscited by 7
  24. 2024GenAI Content Detection Task 2: AI vs. Human -- Academic Essay Authenticity Challengecited by 6
  25. 2025Large Language Models Penetration in Scholarly Writing and Peer Reviewcited by 6
  26. 2025Linguistic and Embedding-Based Profiling of Texts generated by Humans and Large Language Modelscited by 6
  27. 2025Diversity Boosts AI-Generated Text Detectioncited by 6
  28. 2026The Impact of AI-Generated Text on the Internetcited by 6
  29. 2024Synthetic Lyrics Detection Across Languages and Genrescited by 4
  30. 2024LLM Detectors Still Fall Short of Real World: Case of LLM-Generated Short News-Like Postscited by 4
  31. 2024Mitigating Paraphrase Attacks on Machine-Text Detectors via Paraphrase Inversioncited by 4
  32. 2025A Training-free Method for LLM Text Attributioncited by 4
  33. 2025How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Studycited by 4
  34. 2026StealthRL: Reinforcement Learning Paraphrase Attacks for Multi-Detector Evasion of AI-Text Detectorscited by 4
  35. 2024Towards Reliable Detection of LLM-Generated Texts: A Comprehensive Evaluation Framework with CUDRTcited by 3
  36. 2024MOSAIC: Multiple Observers Spotting AI Contentcited by 3
  37. 2025Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detectioncited by 3
  38. 2025Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detectioncited by 3

50 most cited of 158 All 158 (JSON)