# Unsupervised and Distributional Detection of Machine-Generated Text

arXiv: https://arxiv.org/abs/2111.02878 · HTML: https://mlsanity.com/paper/2111.02878/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 4 Nov 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [Automatic Detection of Machine Generated Text: A Critical Survey](https://mlsanity.com/paper/2011.01314/) (2020)
- [TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://mlsanity.com/paper/2109.13296/) (2021)
- [A Distributional Approach to Controlled Text Generation](https://mlsanity.com/paper/2012.11635/) (2020)
- [RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text](https://mlsanity.com/paper/2010.03070/) (2020)
- [How Decoding Strategies Affect the Verifiability of Generated Text](https://mlsanity.com/paper/1911.03587/) (2019)
- [Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation](https://mlsanity.com/paper/1909.11974/) (2019)
- [F^2-Softmax: Diversifying Neural Text Generation via Frequency Factorized Softmax](https://mlsanity.com/paper/2009.09417/) (2020)
- [Identifying Automatically Generated Headlines using Transformers](https://mlsanity.com/paper/2009.13375/) (2020)

## Built on by (most cited first, 23 of 23)

- [On the Reliability of Watermarks for Large Language Models](https://mlsanity.com/paper/2306.04634/) (2023)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods](https://mlsanity.com/paper/2210.07321/) (2022)
- [Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective](https://mlsanity.com/paper/2210.10488/) (2022)
- [GPT-who: An Information Density-based Machine-Generated Text Detector](https://mlsanity.com/paper/2310.06202/) (2023)
- [Few-Shot Detection of Machine-Generated Text using Style Representations](https://mlsanity.com/paper/2401.06712/) (2024)
- [Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian](https://mlsanity.com/paper/2206.01583/) (2022)
- [Generative AI Text Classification using Ensemble LLM Approaches](https://mlsanity.com/paper/2309.07755/) (2023)
- [Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts?](https://mlsanity.com/paper/2304.01002/) (2023)
- [Survey on AI-Generated Media Detection: From Non-MLLM to MLLM](https://mlsanity.com/paper/2502.05240/) (2025)
- [Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection](https://mlsanity.com/paper/2402.00412/) (2024)
- [A Survey on Responsible Generative AI: What to Generate and What Not](https://mlsanity.com/paper/2404.05783/) (2024)
- [GenAI Content Detection Task 2: AI vs. Human -- Academic Essay Authenticity Challenge](https://mlsanity.com/paper/2412.18274/) (2024)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [Alteration-free and Model-agnostic Origin Attribution of Generated Images](https://mlsanity.com/paper/2305.18439/) (2023)
- [Stacking the Odds: Transformer-Based Ensemble for AI-Generated Text Detection](https://mlsanity.com/paper/2310.18906/) (2023)
- [AI-generated Text Detection: A Multifaceted Approach to Binary and Multiclass Classification](https://mlsanity.com/paper/2505.11550/) (2025)
- [Synthetic Text Detection: Systemic Literature Review](https://mlsanity.com/paper/2210.06336/) (2022)
- [Detecting AI-Generated Texts in Cross-Domains](https://mlsanity.com/paper/2410.13966/) (2024)
- [MasonTigers at SemEval-2024 Task 8: Performance Analysis of Transformer-based Models on Machine-Generated Text Detection](https://mlsanity.com/paper/2403.14989/) (2024)
- [Measuring Human Involvement in AI-Generated Text: A Case Study on Academic Writing](https://mlsanity.com/paper/2506.03501/) (2025)
- [Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift](https://mlsanity.com/paper/2606.25152/) (2026)
- [Beyond Perplexity: Character Distribution Signatures and the MDTA Benchmark for AI Text Detection](https://mlsanity.com/paper/2605.01647/) (2026)

## Abstract (arXiv)

The power of natural language generation models has provoked a flurry of interest in automatic methods to detect if a piece of text is human or machine-authored. The problem so far has been framed in a standard supervised way and consists in training a classifier on annotated data to predict the origin of one given new document. In this paper, we frame the problem in an unsupervised and distributional way: we assume that we have access to a large collection of unannotated documents, a big fraction of which is machine-generated. We propose a method to detect those machine-generated documents leveraging repeated higher-order n-grams, which we show over-appear in machine-generated text as compared to human ones. That weak signal is the starting point of a self-training setting where pseudo-labelled documents are used to train an ensemble of classifiers. Our experiments show that leveraging that signal allows us to rank suspicious documents accurately. Precision at 5000 is over 90% for top-k sampling strategies, and over 80% for nucleus sampling for the largest model we used (GPT2-large). The drop with increased size of model is small, which could indicate that the results hold for other current and future large language models.