# You Can Tell Who's Asking: What the Web's Questions Are Made Of, and Where They Come From

arXiv: https://arxiv.org/abs/2609.24106 · HTML: https://mlsanity.com/paper/2609.24106/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Builds on (most cited first, 25 of 25)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://mlsanity.com/paper/2102.07033/) (2021)
- [What's In My Big Data?](https://mlsanity.com/paper/2310.20707/) (2023)
- [GEO: Generative Engine Optimization](https://mlsanity.com/paper/2311.09735/) (2023)
- [GooAQ: Open Question Answering with Diverse Answer Types](https://mlsanity.com/paper/2104.08727/) (2021)
- [A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism](https://mlsanity.com/paper/2401.05749/) (2024)
- [Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media](https://mlsanity.com/paper/2412.18148/) (2024)
- [ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search](https://mlsanity.com/paper/2006.05324/) (2020)
- [Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow](https://mlsanity.com/paper/2307.07367/) (2023)
- [CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training](https://mlsanity.com/paper/2110.07731/) (2021)
- [WebFAQ: A Multilingual Collection of Natural Q&A Datasets for Dense Retrieval](https://mlsanity.com/paper/2502.20936/) (2025)
- [The Impact of AI-Generated Text on the Internet](https://mlsanity.com/paper/2604.26965/) (2026)
- [Retrieval Collapses When AI Pollutes the Web](https://mlsanity.com/paper/2602.16136/) (2026)
- [Improved methodology for longitudinal Web analytics using Common Crawl](https://mlsanity.com/paper/2404.09770/) (2024)
- [Stack Overflow Is Not Dead Yet: Crowd Answers Still Matter](https://mlsanity.com/paper/2509.05879/) (2025)
- [WebFAQ 2.0: A Multilingual QA Dataset with Mined Hard Negatives for Dense Retrieval](https://mlsanity.com/paper/2602.17327/) (2026)
- [DeGenTWeb: A First Look at LLM-dominant Websites](https://mlsanity.com/paper/2605.00087/) (2026)

## Abstract (arXiv)

Questions scraped from the web are used across academia and industry as a proxy for what people want to know. Across QA training data, retrieval benchmarks, and content strategy, questions on a page are assumed to reflect human intent. We test this assumption at scale by extracting 13.4B question occurrences across 110 FineWeb snapshots (2013-2025), and report three findings. First, you can tell who is asking: provenance (the host/page of questions) leaves a signal in question form, and a logistic model can separate genuine user questions from templated/manufactured ones at AUC 0.725 via length and surrounding context rather than question type, though only 0.554 against commerce FAQ writing. Second, question frequency does not measure demand: the most-frequent questions are boilerplate/templated (over 70% of the top thousand), so occurrence counts measure how often a string was published and not how often it was asked. Third, over twelve years the genuine share of occurrences fell by 79% (42-56% after controlling for crawl composition), with question length and context decreasing. We present the first diachronic, occurrence-level measurement of web question provenance, and find the crawlable web's questions have shifted from being asked by humans toward manufactured for machines to read.