mlsanityData to 14 Oct 2026

Phase Transitions in the Output Distribution of Large Language Models

arXiv 2405.17088 ↗First posted 27 May 2024cs.LG cond-mat.stat-mech cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

In a physical system, changing parameters such as temperature can induce a phase transition: an abrupt change from one state of matter to another. Analogous phenomena have recently been observed in large language models. Typically, the task of identifying phase transitions requires human analysis and some prior understanding of the system to narrow down which low-dimensional properties to monitor and analyze. Statistical methods for the automated detection of phase transitions from data have recently been proposed within the physics community. These methods are largely system agnostic and, as shown here, can be adapted to study the behavior of large language models. In particular, we quantify distributional changes in the generated output via statistical distances, which can be efficiently estimated with access to the probability distribution over next-tokens. This versatile approach is capable of discovering new phases of behavior and unexplored transitions -- an ability that is particularly exciting in light of the rapid development of language models and their emergent capabilities.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 59 of its 118 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

59 of 118 references.

References · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2021Program Synthesis with Large Language Modelscited by 3,637
  7. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  8. 2022Emergent Abilities of Large Language Modelscited by 2,512
  9. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  10. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  11. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  12. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
38 more
  1. 2021Scaling Vision Transformerscited by 999
  2. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  3. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  5. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  6. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  7. 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
  8. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
  9. 2020Scaling Laws for Autoregressive Generative Modelingcited by 522
  10. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
  11. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 328
  12. 2023Language Models Represent Space and Timecited by 316
  13. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 285
  14. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  15. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 230
  16. 2018Gradient Descent Happens in a Tiny Subspacecited by 217
  17. 2019A Constructive Prediction of the Generalization Error Across Scalescited by 210
  18. 2022Predictability and Surprise in Large Generative Modelscited by 210
  19. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 156
  20. 2022Omnigrok: Grokking Beyond Algorithmic Datacited by 146
  21. 2023Pretraining task diversity and the emergence of non-Bayesian in-context learning for regressioncited by 146
  22. 2023The Quantization Model of Neural Scalingcited by 138
  23. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  24. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 124
  25. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  26. 2023A Theory for Emergence of Complex Skills in Language Modelscited by 113
  27. 2022Broken Neural Scaling Lawscited by 109
  28. 2022The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenoncited by 94
  29. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
  30. 2017Critical Learning Periods in Deep Neural Networkscited by 79
  31. 2021Probing Across Time: What Does RoBERTa Know and When?cited by 73
  32. 2016Discovering Phase Transitions with Unsupervised Learningcited by 49
  33. 2023Grokking as a First Order Phase Transition in Two Layer Networkscited by 48
  34. 2017Unsupervised learning of phase transitions: from principal component analysis to variational autoencoderscited by 47
  35. 2023On the Stepwise Nature of Self-Supervised Learningcited by 36
  36. 2024A phase transition between positional and semantic learning in a solvable model of dot-product attentioncited by 30
  37. 2023Grokking in Linear Estimators -- A Solvable Model that Groks without Understandingcited by 25
  38. 2019Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshopcited by 22

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 3.

7 papers.

  1. 2025Model Organisms for Emergent Misalignmentcited by 91
  2. 2024Phase transition in large language models and the criticality of natural languagescited by 13
  3. 2025Decomposing Behavioral Phase Transitions in LLMs: Order Parameters for Emergent Misalignmentcited by 5
  4. 2026From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretabilitycited by 4
  5. 2025Another Turn, Better Output? A Turn-Wise Analysis of Iterative LLM Promptingcited by 3
  6. 2026What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the modelcited by 3
  7. 2026Temperature-driven inversion and nonlinear dynamics in ChatGPT-like AIscited by 1