mlsanityData to 14 Oct 2026

Phase Transitions in Driven Informational Systems: A Two-Field Perspective on Learning Theory and Non-Equilibrium Chemistry

arXiv 2605.16325 ↗First posted 5 May 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Phase-transition phenomena in deep learning (grokking, emergent capabilities, and ontological reorganization under context shift) have been studied through several lenses, including representational compression, singular learning theory, and information-theoretic progress measures. Independently, non-equilibrium statistical physics has identified phase transitions in driven chemical reaction networks underlying prebiotic selection, with empirical signatures that are difficult to reproduce within single-field gradient accounts. We propose a perspective in which both classes of phenomena admit a common description as driven informational systems: stochastic processes governed by two gradient fields, an entropy production rate Sigma and an information quasi-potential Phi_I := -ln p*, where p* is the stationary density. Within this framework we introduce two candidate order parameters: an adversarial breakdown threshold alpha_dagger and a self-referential coupling threshold kappa_c. The joint scaling of (alpha_dagger, kappa_c) defines a candidate universality class with exponents (gamma_1, gamma_2). We outline the geometric structure of this framework, identify falsifiable predictions distinguishing it from single-field alternatives, and show consistency with recent empirical findings (2024--2026) on alignment transitions, adversarial breakdown scaling, and partial introspection in large language models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 72 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

32 of 72 references.

References · 32

  1. 2022Emergent Abilities of Large Language Modelscited by 2,512
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2022In-context Learning and Induction Headscited by 964
  4. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  5. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  6. 2018MINE: Mutual Information Neural Estimationcited by 683
  7. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  8. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 230
  9. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 113
  10. 2025Model Organisms for Emergent Misalignmentcited by 91
  11. 2026Emergent Introspective Awareness in Large Language Modelscited by 82
  12. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samplescited by 72
20 more
  1. 2025Convergent Linear Representations of Emergent Misalignmentcited by 71
  2. 2024Loss Landscape Degeneracy and Stagewise Development in Transformerscited by 40
  3. 2026Emergent Misalignment is Easy, Narrow Misalignment is Hardcited by 23
  4. 2024The Complexity Dynamics of Grokkingcited by 22
  5. 2025Privileged Self-Access Matters for Introspection in AIcited by 22
  6. 2024Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transitioncited by 17
  7. 2025You Are What You Eat -- AI Alignment Requires Understanding How Data Shapes Structure and Generalisationcited by 17
  8. 2022On Optimal Learning Under Targeted Data Poisoningcited by 14
  9. 2023Grokking as Compression: A Nonlinear Complexity Perspectivecited by 14
  10. 2025Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerabilitycited by 7
  11. 2026Emergent Introspection in AI is Content-Agnosticcited by 7
  12. 2026Mechanisms of Introspective Awarenesscited by 7
  13. 2024Unexpected Benefits of Self-Modeling in Neural Systemscited by 5
  14. 2025Decomposing Behavioral Phase Transitions in LLMs: Order Parameters for Emergent Misalignmentcited by 5
  15. 2026From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretabilitycited by 4
  16. 2026Grokking as a Falsifiable Finite-Size Transitioncited by 3
  17. 2024Biological arrow of time: Emergence of tangled information hierarchies and self-modelling dynamicscited by 2
  18. 2025Agnostic Learning under Targeted Poisoning: Optimal Rates and the Role of Randomnesscited by 2
  19. 2026Norm-Hierarchy Transitions in Representation Learning: When and Why Neural Networks Abandon Shortcutscited by 2
  20. 2024Active Inference for Self-Organizing Multi-LLM Systems: A Bayesian Thermodynamic Approach to Adaptationcited by 1

Built on by

Not cited yet.