Silent Collapse in Recursive Learning Systems
Not on the map. Why?
Abstract (arXiv)
Recursive learning -- where models are trained on data generated by previous versions of themselves -- is increasingly common in large language models, autonomous agents, and self-supervised systems. However, standard performance metrics (loss, perplexity, accuracy) often fail to detect internal degradation before it becomes irreversible. Here we identify a phenomenon we call silent collapse: under broad recursive conditions, model internal distributions -- predictive entropy, representational diversity, and tail coverage -- progressively contract even as conventional metrics appear stable or improving. We discover that silent collapse is not abrupt. Its onset is reliably preceded by three trajectory-level precursors: (1) contraction of anchor entropy, (2) freezing of representation drift, and (3) erosion of tail coverage. These signals manifest multiple generations before any degradation in standard validation metrics, enabling early warning. Based on these precursors, we propose the MTR (Monitor--Trust--Regulator) framework, a lightweight metacognitive loop that monitors trajectory statistics, estimates a slow-timescale trust variable, and adaptively modulates the effective learning intensity. MTR provides early warning and actively prevents silent collapse without requiring access to pristine real data -- a critical advantage when original data is unavailable, contaminated, or private.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 14 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
6 of 14 references.
References · 6
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2016Concrete Problems in AI Safetycited by 1,642
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
- 2018Scalable agent alignment via reward modeling: a research directioncited by 447
Built on by
Not cited yet.