Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies
On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training
Abstract (arXiv)
Successive self-training on a language model's own outputs is widely characterized as a process of flattening: diversity drops, distributions narrow, and the text becomes "more like itself." We provide evidence that this characterization is incomplete. Across eleven generations of self-training on five models (GPT-2 124M, Pythia-410M, Pythia-1.4B, OPT-1.3B, Pythia-2.8B), language is not flattened uniformly -- it is restructured. Surface markers (discourse connectives, hedges, em-dashes) rise, while mid- and deep-syntactic structures (questions, parentheticals, passives, subjunctives) collapse. We formalize this asymmetric collapse as the Structural Depth Hypothesis (SDH): the per-generation decay rate of a linguistic feature is predicted primarily by its structural depth -- the number of nested syntactic dependencies it requires -- and only secondarily by its generation-zero output frequency. Pooling 17-feature panels from five models spanning three architecture families (N=85), the pooled Spearman correlation is rho=0.540 (p < 10^{-6}; cluster-bootstrap 95% CI [0.434, 0.634]), while frequency is a substantially weaker predictor (rho=0.225). A matched human-text fine-tuning control yields rho=0.039 (p=0.88), confirming the gradient is self-training-specific. We further document a Superficial Complexity Paradox: aggregate complexity proxies (dep-tree depth, TTR, word length) all rise as the underlying clause structure dies, with direct implications for training-data curation and LLM-text detection.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 15 of its 30 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
15 of 30 references.
References · 15
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
- 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
- 2019Neural Text Generation with Unlikelihood Trainingcited by 503
- 2023Self-Consuming Generative Models Go MADcited by 250
- 2023Does Writing with Language Models Reduce Content Diversity?cited by 192
- 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
- 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 151
- 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
- 2024A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'cited by 115
- 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 79
- 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 79
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Break Step: Recursive Training Resonates with Replayed Sampling Noise
- 2026Not All Synthetic Data Is Yours to Learn From
- 2024Self-Improvement in Language Models: The Sharpening Mechanism
- 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
- 2024Collapse of Self-trained Language Models
- 2024Revisiting the Superficial Alignment Hypothesis
- 2026From Context Shift to Stylistic Collapse: Why Training Objectives Matter More Than Scale
- 2026Exposure is Optional: Learning Unlike Coordination in Language Models