# Entropy Collapse: A Universal Failure Mode of Intelligent Systems

arXiv: https://arxiv.org/abs/2512.12381 · HTML: https://mlsanity.com/paper/2512.12381/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 43 references that are in the map.
- First posted on arXiv 13 Dec 2025.

## Builds on (most cited first, 8 of 8)

- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data](https://mlsanity.com/paper/2404.01413/) (2024)
- [A Tale of Tails: Model Collapse as a Change of Scaling Laws](https://mlsanity.com/paper/2402.07043/) (2024)

## Built on by (most cited first, 2 of 2)

- [Spectral Entropy Collapse as a Phase Transition in Delayed Generalisation: An Interventional and Predictive Framework for Grokkin](https://mlsanity.com/paper/2604.13123/) (2026)
- [Dynamic Intelligence Ceilings: Measuring Long-Horizon Limits of Planning and Creativity in Artificial Systems](https://mlsanity.com/paper/2601.06102/) (2026)

## Abstract (arXiv)

A foundational assumption in complex-system collapse studies is that critical transitions are second-order, preceded by early-warning signals like rising autocorrelation, variance, and critical slowing down (Scheffer, 2009). We show this fails for feedback-amplified adaptive systems. We prove entropy collapse - the irreversible contraction of effective state space when feedback amplification alpha exceeds novelty regeneration beta - is a first-order (discontinuous) phase transition. Four exact results: (1) Threshold alpha_c(beta) = 1/(1-beta), from Jacobian spectrum of Multiplicative-Weights operator. (2) Discontinuity: entropy order parameter m = 1 - H_ss/H_max jumps Delta m_0 = 0.698 at alpha_c, with hysteresis Delta H_hyst approx 2.73 nats (lower bound; up to 3.9 nats in simulations); no pre-transition warnings as autocorrelation and variance stay finite. (3) Relaxation exponent nu = 1, from transcritical bifurcation (R^2 = 0.9997 vs. simulation); universality across update mechanisms. (4) Two classes: feedback curvature kappa = f''(1/N) determines order - Class 1 (kappa > 0, convex, e.g., power-law) irreversible with nu = 1; Class 2 (kappa = 0, linear) reversible with nu = 1/2. Theorems validated in neural experiments on two-layer autoregressive transformer (SmallGPT, N=50 vocab, 92 conditions, 8 seeds/condition): Delta H_hyst^NN = 2.92 nats > 2.73 (Theorem 2); nu^NN = 1.14 +/- 0.13, R^2 = 0.977 (Theorem 3). This unifies AI model collapse (Shumailov et al., 2023), economic institutional sclerosis, and evolutionary genetic bottlenecks as first-order entropy-driven processes, evading standard early-warning monitoring.