The Economics of Model Collapse: Equilibrium, Welfare, and Optimal Provenance Subsidies in Synthetic Data Markets
On the map: Synthetic data and self-improvement for LLMs → Model collapse from synthetic data
Abstract (arXiv)
Generative artificial intelligence is rapidly transforming the supply side of training data: an increasing share of new tokens, images, and structured records is produced by previous-generation models rather than by human originators. Recursive training on such synthetic content induces a measurable and often irreversible loss of distributional fidelity, a phenomenon known as model collapse. We develop the first unified microeconomic theory of synthetic data markets under model collapse. We introduce the Synthetic Data Contamination Equilibrium (SDCE), prove existence and generic uniqueness, derive a welfare decomposition W = W_prod + W_cons - L_coll - L_info, establish a Wasserstein-gradient-flow mean-field collapse limit, prove an impossibility of information-constrained implementation, and obtain closed-form expressions for the welfare-maximizing provenance subsidy s* = KL(q||p)/(2 kappa) and the welfare-maximizing watermark strength w* = (1 - psi) KL(q||p)/(2 kappa psi). We prove an information-theoretic Cramer-Rao lower bound on any provenance estimator using only producer-side observations and show that the Provenance-Market Iterative Retraining (PMIR) algorithm attains this bound up to constants while converging to an epsilon-SDCE in O(epsilon^-2 log T) iterations. A reduced-form OLS estimation on a C4-synthetic benchmark over ten retraining generations yields a collapse-rate coefficient b-hat = 0.181 (HAC s.e. 0.024), within one standard error of the structural prediction 0.183. Calibrated experiments raise generation-ten model quality by 23.1 percent over the unregulated benchmark while lowering the 2-Wasserstein drift on a held-out diversity probe from 0.318 to 0.142. Scaling experiments over generations t in {1,...,10} recover a logarithmic-in-t collapse law log Q_t = log Q_0 - 0.183 t rho^2 with R^2 = 0.962.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Model collapse from synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 20 of its 39 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
20 of 39 references.
Closest ancestors · 2
Same direction, newest first
- 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data1 Apr
- 2024A Tale of Tails: Model Collapse as a Change of Scaling Laws10 Feb
Other references · 18
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2018Computational Optimal Transportcited by 780
- 2022Quantifying Memorization Across Neural Language Modelscited by 763
- 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
- 2023A Watermark for Large Language Modelscited by 725
- 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
6 more
- 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
- 2023Self-Consuming Generative Models Go MADcited by 250
- 2023Foundation Models and Fair Usecited by 116
- 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 100
- 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 79
- 2023Combining Generative Artificial Intelligence (AI) and the Internet: Heading towards Evolution or Degradation?cited by 41
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026When Sample Selection Bias Precipitates Model Collapse
- 2026The Price of Ignorance: Information-Free Quotation for Data Retention in Machine Unlearning
- 2025When Models Don't Collapse: On the Consistency of Iterative MLE
- 2024Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
- 2025A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective
- 2026From Collapse to Improvement: Statistical Perspectives on the Evolutionary Dynamics of Iterative Training on Contaminated Sources
- 2026Reviewing Model Collapse and Countermeasures
- 2026Human-Provenance Verification should be Treated as Labor Infrastructure in AI-Saturated Markets