# Phase transitions for the noisy transformer model in arbitrary dimension

arXiv: https://arxiv.org/abs/2606.05140 · HTML: https://mlsanity.com/paper/2606.05140/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [A mathematical perspective on Transformers](https://mlsanity.com/paper/2312.10794/) (2023)
- [The emergence of clusters in self-attention dynamics](https://mlsanity.com/paper/2305.05465/) (2023)
- [Sinkformers: Transformers with Doubly Stochastic Attention](https://mlsanity.com/paper/2110.11773/) (2021)
- [Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View](https://mlsanity.com/paper/1906.02762/) (2019)
- [Redesigning the Transformer Architecture with Insights from Multi-particle Dynamical Systems](https://mlsanity.com/paper/2109.15142/) (2021)
- [On the Structure of Stationary Solutions to McKean-Vlasov Equations with Applications to Noisy Transformers](https://mlsanity.com/paper/2510.20094/) (2025)
- [Phase transitions in Doi-Onsager, Noisy Transformer, and other multimodal models](https://mlsanity.com/paper/2604.16288/) (2026)

## Built on by (most cited first, 1 of 1)

- [On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers](https://mlsanity.com/paper/2607.18584/) (2026)

## Abstract (arXiv)

We study the McKean--Vlasov free energy on the unit sphere associated with the unnormalized self-attention (USA) model for noisy transformer dynamics. We prove a sharp global-minimizer dichotomy in every dimension d≥2. There is a unique β_*(d)>0 such that beginequation* fracId/2+1(β_*(d))Id/2(β_*(d))=frac1d, endequation* where I_nu is the modified Bessel function of the first kind. For 0<β≤ β_*(d), the uniform density remains the unique global minimizer up to the linear-stability threshold beginequation* K_#(d)(β)=fracβd/22d/2Gamma(d/2)Id/2(β), endequation* and the phase transition is continuous. For β>β_*(d), the uniform density is not globally minimizing at K_#(d)(β), so the critical coupling satisfies Kc<K_#(d)(β) and the transition is discontinuous. This result generalizes the authors' recent d=2 work arXiv:2604.16288 to arbitrary dimension. The proof uses the sharp Beckner--Onofri/logarithmic Hardy-Littlewood-Sobolev (HLS) inequality on the sphere, together with a Funk--Hecke/Bessel coefficient computation and a degree-two quartic obstruction.