You Can Tell Who's Asking: What the Web's Questions Are Made Of, and Where They Come From
Not on the map. Why?
Abstract (arXiv)
Questions scraped from the web are used across academia and industry as a proxy for what people want to know. Across QA training data, retrieval benchmarks, and content strategy, questions on a page are assumed to reflect human intent. We test this assumption at scale by extracting 13.4B question occurrences across 110 FineWeb snapshots (2013-2025), and report three findings. First, you can tell who is asking: provenance (the host/page of questions) leaves a signal in question form, and a logistic model can separate genuine user questions from templated/manufactured ones at AUC 0.725 via length and surrounding context rather than question type, though only 0.554 against commerce FAQ writing. Second, question frequency does not measure demand: the most-frequent questions are boilerplate/templated (over 70% of the top thousand), so occurrence counts measure how often a string was published and not how often it was asked. Third, over twelve years the genuine share of occurrences fell by 79% (42-56% after controlling for crawl composition), with question length and context decreasing. We present the first diachronic, occurrence-level measurement of web question provenance, and find the crawlable web's questions have shifted from being asked by humans toward manufactured for machines to read.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 57 references that are in the map. First posted on arXiv 21 Sep 2026.
Builds on
25 of 57 references.
References · 25
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
- 2018Datasheets for Datasetscited by 1,386
- 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
- 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
- 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 461
- 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
- 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
- 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
- 2023What's In My Big Data?cited by 129
- 2023GEO: Generative Engine Optimizationcited by 56
13 more
- 2021GooAQ: Open Question Answering with Diverse Answer Typescited by 53
- 2024A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelismcited by 35
- 2024Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Mediacited by 35
- 2020ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Searchcited by 34
- 2023Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflowcited by 22
- 2021CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Trainingcited by 13
- 2025WebFAQ: A Multilingual Collection of Natural Q&A Datasets for Dense Retrievalcited by 10
- 2026The Impact of AI-Generated Text on the Internetcited by 6
- 2026Retrieval Collapses When AI Pollutes the Webcited by 4
- 2024Improved methodology for longitudinal Web analytics using Common Crawlcited by 3
- 2025Stack Overflow Is Not Dead Yet: Crowd Answers Still Mattercited by 3
- 2026WebFAQ 2.0: A Multilingual QA Dataset with Mined Hard Negatives for Dense Retrievalcited by 1
- 2026DeGenTWeb: A First Look at LLM-dominant Websitescited by 1
Built on by
Not cited yet.