# Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study

arXiv: https://arxiv.org/abs/2008.13533 · HTML: https://mlsanity.com/paper/2008.13533/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2020.

## Built on by (most cited first, 4 of 4)

- [Artificial Text Detection via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2109.04825/) (2021)
- [UnMASKed: Quantifying Gender Biases in Masked Language Models through Linguistically Informed Job Market Prompts](https://mlsanity.com/paper/2401.15798/) (2024)
- [Improving Detection of Watermarked Language Models](https://mlsanity.com/paper/2508.13131/) (2025)
- [Classification of worldwide news articles by perceived quality, 2018-2024](https://mlsanity.com/paper/2511.16416/) (2025)

## Abstract (arXiv)

Large generative language models such as GPT-2 are well-known for their ability to generate text as well as their utility in supervised downstream tasks via fine-tuning. Our work is twofold: firstly we demonstrate via human evaluation that classifiers trained to discriminate between human and machine-generated text emerge as unsupervised predictors of "page quality", able to detect low quality content without any training. This enables fast bootstrapping of quality indicators in a low-resource setting. Secondly, curious to understand the prevalence and nature of low quality pages in the wild, we conduct extensive qualitative and quantitative analysis over 500 million web articles, making this the largest-scale study ever conducted on the topic.