SlopShape: Identifying AI-Generated Commercial Web Content
Not on the map. Why?
Abstract (arXiv)
Word-level detectors identify unedited AI-generated text almost perfectly, but the literature documents their brittleness under rewording, and a word-level score neither characterizes a text nor identifies which AI model wrote it. We ask whether AI-generated text can be identified one level deeper, from structural signatures: how information is presented, in what order, with what evidence, and in what voice. We replicate StoryScope (Russell et al., 2026), which showed such patterns for AI-generated fiction, on commercial content: 2,250 pre-ChatGPT human blog posts from 268 company domains against 11,250 AI mirrors from five frontier models. A 203-feature instrument, applied by an LLM and validated in a human gold-annotation session (human-human kappa 0.939, human-model 0.951), detects AI posts from its 176 structural features alone at 97.0 macro-F1 on held-out companies, nearly unchanged (96.1) when every AI post is reworded by its own model. The signal characterizes and attributes: AI posts share a tidy, self-announcing shape, 68.6% are attributed to the correct source against a 16.7% chance rate, and human posts occupy rare structural configurations. All effects replicate StoryScope's, consistent in direction and at least as large in magnitude. We release pipeline, instrument, prompts, code, and aggregate artifacts.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 53 references that are in the map. First posted on arXiv 14 Sep 2026.
Builds on
19 of 53 references.
References · 19
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
- 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 429
- 2023Can AI-Generated Text be Reliably Detected?cited by 386
- 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 263
- 2024RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectorscited by 163
- 2023Testing of Detection Tools for AI-Generated Textcited by 90
- 2024Delving into LLM-assisted writing in biomedical publications through excess vocabularycited by 78
- 2025The Widespread Adoption of Large Language Model-Assisted Writing Across Societycited by 51
- 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
7 more
- 2024The Rise of AI-Generated Content in Wikipediacited by 22
- 2025DAMAGE: Detecting Adversarially Modified AI Generated Textcited by 20
- 2025Generative Engine Optimization: How to Dominate AI Searchcited by 18
- 2025NarraBench: A Comprehensive Framework for Narrative Benchmarkingcited by 13
- 2025AI use in American newspapers is widespread, uneven, and rarely disclosedcited by 10
- 2026Impact of AI Search Summaries on Website Traffic: Evidence from Google AI Overviews and Wikipediacited by 4
- 2026Diagnosing and Repairing Citation Failures in Generative Engine Optimizationcited by 3
Built on by
Not cited yet.