# Spotlights and Blindspots: Evaluating Machine-Generated Text Detection

arXiv: https://arxiv.org/abs/2604.16607 · HTML: https://mlsanity.com/paper/2604.16607/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 17 Apr 2026.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [RADAR: Robust AI-Text Detection via Adversarial Learning](https://mlsanity.com/paper/2307.03838/) (2023)
- [RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors](https://mlsanity.com/paper/2405.07940/) (2024)
- [MAGE: Machine-generated Text Detection in the Wild](https://mlsanity.com/paper/2305.13242/) (2023)
- [DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text](https://mlsanity.com/paper/2305.17359/) (2023)
- [TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://mlsanity.com/paper/2109.13296/) (2021)
- [Ghostbuster: Detecting Text Ghostwritten by Large Language Models](https://mlsanity.com/paper/2305.15047/) (2023)
- [Contrasting Linguistic Patterns in Human and LLM-Generated News Text](https://mlsanity.com/paper/2308.09067/) (2023)
- [DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning](https://mlsanity.com/paper/2410.20964/) (2024)
- [Raidar: geneRative AI Detection viA Rewriting](https://mlsanity.com/paper/2401.12970/) (2024)
- [LLMDet: A Third Party Large Language Models Generated Text Detection Tool](https://mlsanity.com/paper/2305.15004/) (2023)
- [Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition](https://mlsanity.com/paper/2002.10361/) (2020)
- [GPT-who: An Information Density-based Machine-Generated Text Detector](https://mlsanity.com/paper/2310.06202/) (2023)
- [HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus](https://mlsanity.com/paper/2309.02731/) (2023)
- [On the Detectability of ChatGPT Content: Benchmarking, Methodology, and Evaluation through the Lens of Academic Writing](https://mlsanity.com/paper/2306.05524/) (2023)
- [Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts?](https://mlsanity.com/paper/2304.01002/) (2023)
- [Are AI Detectors Good Enough? A Survey on Quality of Datasets With Machine-Generated Texts](https://mlsanity.com/paper/2410.14677/) (2024)
- [Assaying on the Robustness of Zero-Shot Machine-Generated Text Detectors](https://mlsanity.com/paper/2312.12918/) (2023)
- [Collaborative Evaluation of Deepfake Text with Deliberation-Enhancing Dialogue Systems](https://mlsanity.com/paper/2503.04945/) (2025)

## Built on by (most cited first, 2 of 2)

- [The Perplexity Trap: When Patent Law Makes Human Writing Look Like AI](https://mlsanity.com/paper/2607.13044/) (2026)
- [ARB: A Matched Authorship-Rewriting Benchmark Dataset for AI-Text Detector Evaluation](https://mlsanity.com/paper/2607.29539/) (2026)

## Abstract (arXiv)

With the rise of generative language models, machine-generated text detection has become a critical challenge. A wide variety of models is available, but inconsistent datasets, evaluation metrics, and assessment strategies obscure comparisons of model effectiveness. To address this, we evaluate 15 different detection models from six distinct systems, as well as seven trained models, across seven English-language textual test sets and three creative human-written datasets. We provide an empirical analysis of model performance, the influence of training and evaluation data, and the impact of key metrics. We find that no single system excels in all areas and nearly all are effective for certain tasks, and the representation of model performance is critically linked to dataset and metric choices. We find high variance in model ranks based on datasets and metrics, and overall poor performance on novel human-written texts in high-risk domains. Across datasets and metrics, we find that methodological choices that are often assumed or overlooked are essential for clearly and accurately reflecting model performance.