Critical dynamics governs deep learning
Not on the map. Why?
Abstract (arXiv)
The rapid advances in artificial intelligence (AI) have largely been driven by scaling deep neural networks (DNNs) - increasing model size, data, and computational resources. Yet performance is ultimately governed by network dynamics. The lack of a principled understanding of DNN dynamics beyond heuristic design has contributed to challenges in robustness, suboptimal performance, high energy consumption, and pathologies in continual and AI-generated content learning. In contrast, the human brain appears largely resilient to these problems, and converging evidence suggests this advantage arises from dynamics poised at a critical phase transition. Inspired by this principle, we propose that criticality provides a unifying framework linking structure, dynamics, and function in DNNs. First, analyzing more than 80 state-of-the-art models, we show that a decade of AI progress has implicitly driven successful networks toward criticality - explaining why some architectures succeeded while others failed. Second, we demonstrate that explicitly incorporating criticality into training improves robustness and accuracy while mitigating key limitations of current models. Third, we show that major AI pathologies - including performance degradation in continual learning and model collapse during training on AI-generated data - reflect a loss of critical dynamics. By maintaining networks near criticality, we provide a principled solution to these failures, demonstrating that criticality-based optimization prevents degradation and collapse. Our results establish criticality as a substrate-independent principle of intelligence, connecting AI progress with fundamental principles of brain function, and offering both theoretical insight and practical strategies to ensure long-term DNN performance and resilience as models scale.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Jul 2025.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.