mlsanityData to 13 Oct 2026

Entropy Collapse: A Universal Failure Mode of Intelligent Systems

arXiv 2512.12381 ↗First posted 13 Dec 2025cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

A foundational assumption in complex-system collapse studies is that critical transitions are second-order, preceded by early-warning signals like rising autocorrelation, variance, and critical slowing down (Scheffer, 2009). We show this fails for feedback-amplified adaptive systems. We prove entropy collapse - the irreversible contraction of effective state space when feedback amplification alpha exceeds novelty regeneration beta - is a first-order (discontinuous) phase transition. Four exact results: (1) Threshold alpha_c(beta) = 1/(1-beta), from Jacobian spectrum of Multiplicative-Weights operator. (2) Discontinuity: entropy order parameter m = 1 - H_ss/H_max jumps Delta m_0 = 0.698 at alpha_c, with hysteresis Delta H_hyst approx 2.73 nats (lower bound; up to 3.9 nats in simulations); no pre-transition warnings as autocorrelation and variance stay finite. (3) Relaxation exponent nu = 1, from transcritical bifurcation (R^2 = 0.9997 vs. simulation); universality across update mechanisms. (4) Two classes: feedback curvature kappa = f''(1/N) determines order - Class 1 (kappa > 0, convex, e.g., power-law) irreversible with nu = 1; Class 2 (kappa = 0, linear) reversible with nu = 1/2. Theorems validated in neural experiments on two-layer autoregressive transformer (SmallGPT, N=50 vocab, 92 conditions, 8 seeds/condition): Delta H_hyst^NN = 2.92 nats > 2.73 (Theorem 2); nu^NN = 1.14 +/- 0.13, R^2 = 0.977 (Theorem 3). This unifies AI model collapse (Shumailov et al., 2023), economic institutional sclerosis, and evolutionary genetic bottlenecks as first-order entropy-driven processes, evading standard early-warning monitoring.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 8 of its 43 references that are in the map. First posted on arXiv 13 Dec 2025.

Builds on

8 of 43 references.

References · 8

  1. 2022Emergent Abilities of Large Language Modelscited by 2,499
  2. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 755
  3. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 716
  4. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 616
  5. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 349
  6. 2023Self-Consuming Generative Models Go MADcited by 246
  7. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 148
  8. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 128

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Spectral Entropy Collapse as a Phase Transition in Delayed Generalisation: An Interventional and Predictive Framework for Grokkincited by 2
  2. 2026Dynamic Intelligence Ceilings: Measuring Long-Horizon Limits of Planning and Creativity in Artificial Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Phase Transition for Budgeted Multi-Agent Synergy
  2. 2025First-Extinction Law for Resampling Processes
  3. 2026World-Model Collapse as a Phase Transition
  4. 2026Intelligence Entropy Principle and the ADE Stability Engineering Framework
  5. 2025Entropy-Reservoir Bregman Projection: An Information-Geometric Unification of Model Collapse
  6. 2026A Markovian View of Iterative-Feedback Loops in Image Generative Models: Neural Resonance and Model Collapse
  7. 2023The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability
  8. 2026Expected Return Causes Outcome-Level Mode Collapse in Reinforcement Learning and How to Fix It with Inverse Probability Scaling