Emergence via Phase Transitions: Mechanism Landscapes and Universal Convergence Across Complex Systems
Not on the map. Why?
Abstract (arXiv)
Across machine learning, biology, and physics, independently evolving systems often converge toward strikingly similar high-level structures despite radically different microscopic details. Grokking circuits converge across random seeds, evolutionary lineages rediscover similar metabolic solutions, and renormalization flows approach common fixed points. We propose the Hierarchical Emergence Framework (HEF) as a candidate universality framework for such convergence phenomena. HEF models emergence as a phase transition in a mechanism landscape constrained by thermodynamic and information-theoretic laws. The framework introduces a critical energy threshold Ec separating an exploration regime with competing mechanisms from a convergence regime governed by a unique minimum-cost mechanism. Under structural assumptions, we prove physical feasibility, derive strict metric contraction, and establish convergence toward a unique fixed-point representation independent of initial conditions. We further connect this convergence structure to causal emergence through Effective Information and mechanism competition entropy. To test the framework, we study delayed generalization ("grokking") in modular arithmetic transformers across 111 experiments. We identify a reproducible empirical fingerprint of the Ec transition: the weight norm peaks systematically before grokking in 92% of runs. Normalized accuracy curves collapse onto a tanh kink (R^2=0.93) consistent with a Landau-Ginzburg universality class, and all grokked models converge to 0.9745+/-0.014 regardless of initialization, weight decay, or training fraction (ANOVA p>0.13). HEF is not presented as a universal theory of emergence, but as a falsifiable mathematical scaffold for studying convergence phenomena across complex systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 56 references that are in the map. First posted on arXiv 25 May 2026.
Builds on
8 of 56 references.
References · 8
- 2022Toy Models of Superpositioncited by 971
- 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
- 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
- 2024The Platonic Representation Hypothesiscited by 382
- 2023To grok or not to grok: Disentangling generalization and memorization on corrupted algorithmic datasetscited by 24
- 2024Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transitioncited by 17
- 2026The Geometry of Multi-Task Grokking: Transverse Instability, Superposition, and Weight Decay Phase Structurecited by 9
- 2026First-Passage Prediction of Grokking Delay: ACalibrated Law under AdamW with Causal Validationcited by 1
Built on by
Not cited yet.