mlsanityData to 13 Oct 2026

Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition

arXiv 2310.06301 ↗First posted 10 Oct 2023cs.LG cs.AIcited by 29

On the map: Mechanistic interpretability → Sparse autoencoder training and evaluation

Abstract (arXiv)

We investigate phase transitions in a Toy Model of Superposition (TMS) using Singular Learning Theory (SLT). We derive a closed formula for the theoretical loss and, in the case of two hidden dimensions, discover that regular k-gons are critical points. We present supporting theory indicating that the local learning coefficient (a geometric invariant) of these k-gons determines phase transitions in the Bayesian posterior as a function of training sample size. We then show empirically that the same k-gon critical points also determine the behavior of SGD training. The picture that emerges adds evidence to the conjecture that the SGD learning trajectory is subject to a sequential learning mechanism. Specifically, we find that the learning process in TMS, be it through SGD or Bayesian learning, can be characterized by a journey through parameter space from regions of high loss and low complexity to regions of low loss and high complexity.

Key facts. Cited by 29 papers in the map (data through 13 Oct 2026). Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability. Builds on 22 of its 38 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

22 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Toy Models of Superposition21 Sep

Other references · 21

  1. 2022Emergent Abilities of Large Language Modelscited by 2,499
  2. 2022In-context Learning and Induction Headscited by 940
  3. 2023Progress measures for grokking via mechanistic interpretabilitycited by 831
  4. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 755
  5. 2019Implicit Regularization in Deep Matrix Factorizationcited by 404
  6. 2017High-dimensional dynamics of generalization error in neural networkscited by 241
  7. 2016Deep Learning without Poor Local Minimacited by 234
  8. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 229
  9. 2018A mathematical theory of semantic development in deep neural networkscited by 184
  10. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 184
  11. 2018A Convergence Analysis of Gradient Descent for Deep Linear Neural Networkscited by 183
  12. 2020Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learningcited by 149
9 more
  1. 2023SGD learning on neural networks: leap complexity and saddle-to-saddle dynamicscited by 142
  2. 2019Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networkscited by 122
  3. 2021Acquisition of Chess Knowledge in AlphaZerocited by 122
  4. 2021Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsitycited by 84
  5. 2019The Implicit Bias of Depth: How Incremental Learning Drives Generalizationcited by 64
  6. 2023The Local Learning Coefficient: A Singularity-Aware Complexity Measurecited by 50
  7. 2020Deep Learning is Singular, and That's Goodcited by 49
  8. 2018Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learningcited by 20
  9. 2020Training Linear Neural Networks: Non-Local Convergence and Complexity Resultscited by 19

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 7; 2025: 12; 2026: 10.

29 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 420
  2. 2024Loss Landscape Degeneracy and Stagewise Development in Transformerscited by 40
  3. 2024Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficientcited by 40
  4. 2024Using Degeneracy in the Loss Landscape for Mechanistic Interpretabilitycited by 13
  5. 2024Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustnesscited by 13
  6. 2025Dynamics of Transient Structure in In-Context Linear Regression Transformerscited by 12
  7. 2025The Loss Kernel: A Geometric Probe for Deep Learning Interpretabilitycited by 9
  8. 2024Estimating the Local Learning Coefficient at Scalecited by 6
  9. 2025Influence Dynamics and Stagewise Data Attributioncited by 6
  10. 2025From Global to Local: A Scalable Benchmark for Local Posterior Samplingcited by 5
  11. 2025Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theorycited by 5
  12. 2026A Basin-Selection Perspective on Grokking via Singular Learning Theorycited by 4
17 more
  1. 2024The Persian Rug: solving toy models of superposition using large-scale symmetriescited by 3
  2. 2025Modes of Sequence Models and Learning Coefficientscited by 3
  3. 2026Stagewise Reinforcement Learning and the Geometry of the Regret Landscapecited by 3
  4. 2025SAFR: Neuron Redistribution for Interpretabilitycited by 2
  5. 2025Grokking vs. Learning: Same Features, Different Encodingscited by 2
  6. 2025Almost Bayesian: The Fractal Dynamics of Stochastic Gradient Descentcited by 2
  7. 2025Emergence of Computational Structure in a Neural Network Physics Simulatorcited by 1
  8. 2026Dead Directions: Geometric Singular Learningcited by 1
  9. 2025Decomposing Prediction Mechanisms for In-Context Recallcited by 0
  10. 2025Using physics-inspired Singular Learning Theory to understand grokking & other phase transitions in modern neural networkscited by 0
  11. 2026Structure and Scale in Simplicial Sequence Modellingcited by 0
  12. 2026Discovering Millions of Interpretable Features with Sparse Autoencoderscited by 0
  13. 2026Local Redundancy: An Information-Theoretic Measure of Plasticity from Synthetic Memorizationcited by 0
  14. 2026The Kinetics of Training: A Driven-Nucleation Rate Law for Emergence, Plasticity Loss, and Circuit Control in Language Modelscited by 0
  15. 2026Langevin dynamics along the zero set of real-analytic potentialscited by 0
  16. 2026Exact Algebraic Computation of Learning Coefficients for Two-Dimensional Singular Modelscited by 0
  17. 2026Emergent phases of superposition: from partial to full representationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Landau theory of quenched criticality in linear in-context learning
  2. 2026Superposition unifies power-law training dynamics
  3. 2024State space models, emergence, and ergodicity: How many parameters are needed for stable predictions?
  4. 2025Learning-at-Criticality in Large Language Models for Quantum Field Theory and Beyond
  5. 2026Towards Physical Intuitions for Alignment Dynamics: A Case Study With Randomness Crystallization
  6. 2023The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability
  7. 2026Generative Modeling of Stochastic Dynamics for Long-Time Evolution
  8. 2026Feature Lottery? A Bifurcation Theory of Concept Emergence