# Decomposing Behavioral Phase Transitions in LLMs: Order Parameters for Emergent Misalignment

arXiv: https://arxiv.org/abs/2508.20015 · HTML: https://mlsanity.com/paper/2508.20015/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 22 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Builds on (most cited first, 18 of 18)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [Persona Vectors: Monitoring and Controlling Character Traits in Language Models](https://mlsanity.com/paper/2507.21509/) (2025)
- [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025)
- [Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs](https://mlsanity.com/paper/2309.07311/) (2023)
- [Model Organisms for Emergent Misalignment](https://mlsanity.com/paper/2506.11613/) (2025)
- [Persona Features Control Emergent Misalignment](https://mlsanity.com/paper/2506.19823/) (2025)
- [Convergent Linear Representations of Emergent Misalignment](https://mlsanity.com/paper/2506.11618/) (2025)
- [Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs](https://mlsanity.com/paper/2502.08640/) (2025)
- [Hermes 3 Technical Report](https://mlsanity.com/paper/2408.11857/) (2024)
- [On the Relationship between Truth and Political Bias in Language Models](https://mlsanity.com/paper/2409.05283/) (2024)
- [Compromising Honesty and Harmlessness in Language Models via Deception Attacks](https://mlsanity.com/paper/2502.08301/) (2025)
- [Hidden Breakthroughs in Language Model Training](https://mlsanity.com/paper/2506.15872/) (2025)
- [Phase transition in large language models and the criticality of natural languages](https://mlsanity.com/paper/2406.05335/) (2024)
- [Machine learning phase transitions: Connections to the Fisher information](https://mlsanity.com/paper/2311.10710/) (2023)
- [Phase Transitions in the Output Distribution of Large Language Models](https://mlsanity.com/paper/2405.17088/) (2024)

## Built on by (most cited first, 5 of 5)

- [LLMs Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactions](https://mlsanity.com/paper/2510.08211/) (2025)
- [From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretability](https://mlsanity.com/paper/2603.29805/) (2026)
- [Emergent Misalignment Can Be Induced by Sycophancy and Reversed via Alignment Gating](https://mlsanity.com/paper/2606.09068/) (2026)
- [Phase Transitions in Driven Informational Systems: A Two-Field Perspective on Learning Theory and Non-Equilibrium Chemistry](https://mlsanity.com/paper/2605.16325/) (2026)
- [See it, Say it, Sorted: Mechanistic Diagnosis and Parameter-Space Mitigation of Emergent Misalignment in LLMs](https://mlsanity.com/paper/2609.34970/) (2026)

## Abstract (arXiv)

Fine-tuning LLMs on narrowly harmful datasets can lead to behavior that is broadly misaligned with respect to human values. To understand when and how this emergent misalignment occurs, we develop a comprehensive framework for detecting and characterizing rapid transitions during fine-tuning using both distributional change detection methods as well as order parameters that are formulated in plain English and evaluated by an LLM judge. Using an objective statistical dissimilarity measure, we quantify how the phase transition that occurs during fine-tuning affects multiple aspects of the model. In particular, we assess what percentage of the total distributional change in model outputs is captured by different aspects, such as alignment or verbosity, providing a decomposition of the overall transition. We also find that the actual behavioral transition occurs later in training than indicated by the peak in the gradient norm alone. Our framework enables the automated discovery and quantification of language-based order parameters, which we demonstrate on examples ranging from knowledge questions to politics and ethics.