mlsanityData to 14 Oct 2026

Epidemiology of Model Collapse: Modeling Synthetic Data Contamination via Bilayer SIR Dynamics

arXiv 2606.05168 ↗First posted 14 Apr 2026cs.CL cs.AI cs.LGcited by 0

On the map: Synthetic data and self-improvement for LLMs → Model collapse from synthetic data

Abstract (arXiv)

Training on synthetic data causes model collapse, but existing analyses treat this as single-chain degradation. In reality, the AI ecosystem involves cross-contamination: models ingest synthetic data from other models, produce new synthetic text, and contaminate shared corpora. We propose a bilayer coupled SIR/SIRS framework -- a phenomenological mean-field model treating data corpora and AI models as two interacting populations, each with susceptible, infected, and recovered compartments linked by cross-layer transmission. The SIRS variant (our primary recommendation) incorporates immunity waning, reflecting that filtered corpora and retrained models remain susceptible to re-contamination. We derive the basic reproduction number R0 = √βD βM / [(γD+μD)(γM+μM)] via the Next Generation Matrix and apply standard epidemic threshold results to the bilayer system. Illustrative scenario-based calibration from public AI text prevalence data yields supercritical dynamics (R0 > 1) across three scenarios; Sobol sensitivity analysis identifies synthetic-text detection as the highest-leverage parameter. A bipartite-network agent-based model confirms mean-field consistency (R2 > 0.96) for dense networks but degrades under heterogeneity. GPT-2 contamination chain experiments (192 runs across WikiText and Shakespeare) show dose-response degradation and diversity loss qualitatively consistent with the threshold picture. Matched-budget source-diversity experiments (1,088 runs) provide suggestive evidence that multi-source mixing modestly attenuates collapse, but the effect vanishes at lower contamination fractions. Intervention analysis identifies detection-based filtering and herd immunity as the highest-leverage strategies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Model collapse from synthetic data, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 11 of its 26 references that are in the map. First posted on arXiv 14 Apr 2026.

Builds on

11 of 26 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Strong Model Collapse7 Oct
  2. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse7 Apr
  3. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data1 Apr

Other references · 8

  1. 2016Pointer Sentinel Mixture Modelscited by 3,096
  2. 2018Model Cards for Model Reportingcited by 1,080
  3. 2023A Watermark for Large Language Modelscited by 725
  4. 2023Self-Consuming Generative Models Go MADcited by 250
  5. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 182
  6. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
  7. 2023The Science of Detecting LLM-Generated Textscited by 128
  8. 2014Epidemic processes in complex networkscited by 58

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Epi-LLM Framework: probing LLM behavioral priors through epidemiological agent-based models
  2. 2026From Collapse to Improvement: Statistical Perspectives on the Evolutionary Dynamics of Iterative Training on Contaminated Sources
  3. 2025When Models Don't Collapse: On the Consistency of Iterative MLE
  4. 2026Position: the Stochastic Parrot in the Coal Mine. Model Collapse is a Threat to Low-Resource Communities
  5. 2025Position: Model Collapse Does Not Mean What You Think
  6. 2026The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data
  7. 2026Driving Epidemic Models with AI Agents: the Epydemix Agent Framework
  8. 2026SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data