# Limits of Detecting Text Generated by Large-Scale Language Models

arXiv: https://arxiv.org/abs/2002.03438 · HTML: https://mlsanity.com/paper/2002.03438/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2020.

## Built on by (most cited first, 15 of 15)

- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [On the Reliability of Watermarks for Large Language Models](https://mlsanity.com/paper/2306.04634/) (2023)
- [TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://mlsanity.com/paper/2109.13296/) (2021)
- [The Gradient of Generative AI Release: Methods and Considerations](https://mlsanity.com/paper/2302.04844/) (2023)
- [Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective](https://mlsanity.com/paper/2210.10488/) (2022)
- [ParaFuzz: An Interpretability-Driven Technique for Detecting Poisoned Samples in NLP](https://mlsanity.com/paper/2308.02122/) (2023)
- [A Benchmark Corpus for the Detection of Automatically Generated Text in Academic Publications](https://mlsanity.com/paper/2202.02013/) (2022)
- [Long-form analogies generated by chatGPT lack human-like psycholinguistic properties](https://mlsanity.com/paper/2306.04537/) (2023)
- [Watermarking Discrete Diffusion Language Models](https://mlsanity.com/paper/2511.02083/) (2025)
- [Stacking the Odds: Transformer-Based Ensemble for AI-Generated Text Detection](https://mlsanity.com/paper/2310.18906/) (2023)
- [Benchmarking of LLM Detection: Comparing Two Competing Approaches](https://mlsanity.com/paper/2406.11670/) (2024)
- [VietBinoculars: A Zero-Shot Approach for Detecting Vietnamese LLM-Generated Text](https://mlsanity.com/paper/2509.26189/) (2025)
- [Deciding When to Switch: E-Processes for Adaptive Minimax Training for Generative Adversarial Nets](https://mlsanity.com/paper/2608.10096/) (2026)
- [VietAIDetector: An Open-Source Zero-Shot Detector for Vietnamese AI-Generated Text](https://mlsanity.com/paper/2608.25478/) (2026)
- [On the Indistinguishability of Human v/s AI Generated Text](https://mlsanity.com/paper/2608.26797/) (2026)

## Abstract (arXiv)

Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns. Here we formulate large-scale language model output detection as a hypothesis testing problem to classify text as genuine or generated. We show that error exponents for particular language models are bounded in terms of their perplexity, a standard measure of language generation performance. Under the assumption that human language is stationary and ergodic, the formulation is extended from considering specific language models to considering maximum likelihood language models, among the class of k-order Markov approximations; error probabilities are characterized. Some discussion of incorporating semantic side information is also given.