# Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore

arXiv: https://arxiv.org/abs/2405.04286 · HTML: https://mlsanity.com/paper/2405.04286/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 75 references that are in the map.
- First posted on arXiv 7 May 2024.

## Builds on (most cited first, 30 of 39)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning](https://mlsanity.com/paper/2312.01552/) (2023)
- [DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2306.05540/) (2023)
- [Automatic Detection of Machine Generated Text: A Critical Survey](https://mlsanity.com/paper/2011.01314/) (2020)
- [Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation](https://mlsanity.com/paper/2304.01746/) (2023)
- [CoEdIT: Text Editing by Task-Specific Instruction Tuning](https://mlsanity.com/paper/2305.09857/) (2023)

## Built on by (most cited first, 27 of 27)

- [DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarios](https://mlsanity.com/paper/2410.23746/) (2024)
- [RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns](https://mlsanity.com/paper/2508.13152/) (2025)
- [DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm](https://mlsanity.com/paper/2509.15550/) (2025)
- [Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry](https://mlsanity.com/paper/2509.01620/) (2025)
- [AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor](https://mlsanity.com/paper/2601.05752/) (2026)
- [DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domains](https://mlsanity.com/paper/2506.06705/) (2025)
- [MoSEs: Uncertainty-Aware AI-Generated Text Detection via Mixture of Stylistics Experts with Conditional Thresholds](https://mlsanity.com/paper/2509.02499/) (2025)
- [WETBench: A Benchmark for Detecting Task-Specific Machine-Generated Text on Wikipedia](https://mlsanity.com/paper/2507.03373/) (2025)
- [Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models](https://mlsanity.com/paper/2508.02360/) (2025)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [DetectRL-X: Towards Reliable Multilingual and Real-World LLM-Generated Text Detection](https://mlsanity.com/paper/2605.15518/) (2026)
- [FOCUS: Forging Originality through Contrastive Use in Self-Plagiarism for Language Models](https://mlsanity.com/paper/2406.00839/) (2024)
- [Towards Reliable Detection of LLM-Generated Texts: A Comprehensive Evaluation Framework with CUDRT](https://mlsanity.com/paper/2406.09056/) (2024)
- [Trends and Challenges in Authorship Analysis: A Review of ML, DL, and LLM Approaches](https://mlsanity.com/paper/2505.15422/) (2025)
- [Beyond the Final Actor: Modeling the Dual Roles of Creator and Editor for Fine-Grained LLM-Generated Text Detection](https://mlsanity.com/paper/2604.04932/) (2026)
- [TSM-Bench: Detecting LLM-Generated Text in Real-World Wikipedia Editing Practices](https://mlsanity.com/paper/2605.31113/) (2026)
- [MAGA-Bench: Machine-Augment-Generated Text via Alignment Detection Benchmark](https://mlsanity.com/paper/2601.04633/) (2026)
- [Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs](https://mlsanity.com/paper/2605.22654/) (2026)
- [DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalities](https://mlsanity.com/paper/2606.04205/) (2026)
- [ARB: A Matched Authorship-Rewriting Benchmark Dataset for AI-Text Detector Evaluation](https://mlsanity.com/paper/2607.29539/) (2026)
- [Code Fingerprints: Disentangled Attribution of LLM-Generated Code](https://mlsanity.com/paper/2603.04212/) (2026)
- [Adversarial Creation and Detection of AI-Generated Social Bot Content](https://mlsanity.com/paper/2606.07219/) (2026)
- [DWT-Fusion: A Signal-Based Framework for Training-Free LLM-Generated Text Detection](https://mlsanity.com/paper/2607.22026/) (2026)
- [Linear Probing Provides Robust and Efficient Detection of Machine-Generated Text](https://mlsanity.com/paper/2608.24780/) (2026)
- [Detecting and Guiding LLM-Generated Korean Poetry with Interpretable Form-level Features](https://mlsanity.com/paper/2608.28986/) (2026)
- [IndicDetect: Evaluating Cross-Lingual LLM-Generated Text Detection for Hindi, Telugu, and Tamil](https://mlsanity.com/paper/2608.29919/) (2026)
- [CrisisFake: Benchmark Validity of AI-Generated Text Detection for Disaster Social Sensing](https://mlsanity.com/paper/2609.35821/) (2026)

## Abstract (arXiv)

The efficacy of detectors for texts generated by large language models (LLMs) substantially depends on the availability of large-scale training data. However, white-box zero-shot detectors, which require no such data, are limited by the accessibility of the source model of the LLM-generated text. In this paper, we propose a simple yet effective black-box zero-shot detection approach based on the observation that, from the perspective of LLMs, human-written texts typically contain more grammatical errors than LLM-generated texts. This approach involves calculating the Grammar Error Correction Score (GECScore) for the given text to differentiate between human-written and LLM-generated text. Experimental results show that our method outperforms current state-of-the-art (SOTA) zero-shot and supervised methods, achieving an average AUROC of 98.62% across XSum and Writing Prompts dataset. Additionally, our approach demonstrates strong reliability in the wild, exhibiting robust generalization and resistance to paraphrasing attacks. Data and code are available at: https://github.com/NLP2CT/GECScore.