# RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors

arXiv: https://arxiv.org/abs/2405.07940 · HTML: https://mlsanity.com/paper/2405.07940/

## Key facts

- Cited by 163 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 95 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 30 of 55)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [RADAR: Robust AI-Text Detection via Adversarial Learning](https://mlsanity.com/paper/2307.03838/) (2023)
- [GPT detectors are biased against non-native English writers](https://mlsanity.com/paper/2304.02819/) (2023)
- [Truncation Sampling as Language Model Desmoothing](https://mlsanity.com/paper/2210.15191/) (2022)
- [M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2305.14902/) (2023)
- [TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://mlsanity.com/paper/2109.13296/) (2021)
- [MGTBench: Benchmarking Machine-Generated Text Detection](https://mlsanity.com/paper/2303.14822/) (2023)

## Built on by (most cited first, 30 of 163)

- [DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively](https://mlsanity.com/paper/2509.26603/) (2025)
- [People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text](https://mlsanity.com/paper/2501.15654/) (2025)
- [The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Models](https://mlsanity.com/paper/2502.11266/) (2025)
- [Beemo: Benchmark of Expert-edited Machine-generated Outputs](https://mlsanity.com/paper/2411.04032/) (2024)
- [RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns](https://mlsanity.com/paper/2508.13152/) (2025)
- [Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods](https://mlsanity.com/paper/2406.15583/) (2024)
- [Is Your Paper Being Reviewed by an LLM? Benchmarking AI Text Detection in Peer Review](https://mlsanity.com/paper/2502.19614/) (2025)
- [DAMAGE: Detecting Adversarially Modified AI Generated Text](https://mlsanity.com/paper/2501.03437/) (2025)
- [Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing](https://mlsanity.com/paper/2502.15666/) (2025)
- [Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement](https://mlsanity.com/paper/2410.14259/) (2024)
- [Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs](https://mlsanity.com/paper/2509.22646/) (2025)
- [MiRAGeNews: Multimodal Realistic AI-Generated News Detection](https://mlsanity.com/paper/2410.09045/) (2024)
- [Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders](https://mlsanity.com/paper/2503.03601/) (2025)
- [HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoring](https://mlsanity.com/paper/2506.02959/) (2025)
- [MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts](https://mlsanity.com/paper/2406.12549/) (2024)
- [Machine-generated text detection prevents language model collapse](https://mlsanity.com/paper/2502.15654/) (2025)
- [DALD: Improving Logits-based Detector without Logits from Black-box LLMs](https://mlsanity.com/paper/2406.05232/) (2024)
- [GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human](https://mlsanity.com/paper/2501.11012/) (2025)
- [AI "News" Content Farms Are Easy to Make and Hard to Detect: A Case Study in Italian](https://mlsanity.com/paper/2406.12128/) (2024)
- [DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning](https://mlsanity.com/paper/2510.17489/) (2025)
- [A Practical Examination of AI-Generated Text Detectors for Large Language Models](https://mlsanity.com/paper/2412.05139/) (2024)
- [DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm](https://mlsanity.com/paper/2509.15550/) (2025)
- [GPTZero: Robust Detection of LLM-Generated Texts](https://mlsanity.com/paper/2602.13042/) (2026)
- [Reliably Bounding False Positives: A Zero-Shot Machine-Generated Text Detection Framework via Multiscaled Conformal Prediction](https://mlsanity.com/paper/2505.05084/) (2025)
- [Detecting LLM-Generated Text with Performance Guarantees](https://mlsanity.com/paper/2601.06586/) (2026)
- [AuthorMist: Evading AI Text Detectors with Reinforcement Learning](https://mlsanity.com/paper/2503.08716/) (2025)
- [OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attribution](https://mlsanity.com/paper/2504.11369/) (2025)
- [Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors](https://mlsanity.com/paper/2505.24523/) (2025)
- [DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domains](https://mlsanity.com/paper/2506.06705/) (2025)
- [textttDroid: A Resource Suite for AI-Generated Code Detection](https://mlsanity.com/paper/2507.10583/) (2025)

## Abstract (arXiv)

Many commercial and open-source models claim to detect machine-generated text with extremely high accuracy (99% or more). However, very few of these detectors are evaluated on shared benchmark datasets and even when they are, the datasets used for evaluation are insufficiently challenging-lacking variations in sampling strategy, adversarial attacks, and open-source generative models. In this work we present RAID: the largest and most challenging benchmark dataset for machine-generated text detection. RAID includes over 6 million generations spanning 11 models, 8 domains, 11 adversarial attacks and 4 decoding strategies. Using RAID, we evaluate the out-of-domain and adversarial robustness of 8 open- and 4 closed-source detectors and find that current detectors are easily fooled by adversarial attacks, variations in sampling strategies, repetition penalties, and unseen generative models. We release our data along with a leaderboard to encourage future research.