mlsanityData to 14 Oct 2026

Synchronization of mean-field models on the circle

arXiv 2507.22857 ↗First posted 30 Jul 2025math.DS cs.LG math.APcited by 12

Not on the map. Why?

Abstract (arXiv)

This paper considers a mean-field model of n interacting particles whose state space is the unit circle, a generalization of the classical Kuramoto model. Global synchronization is said to occur if after starting from almost any initial state, all particles coalesce to a common point on the circle. We propose a general synchronization criterion in terms of L1-norm of the third derivative of the particle interaction function. As an application we resolve a conjecture for the so-called self-attention dynamics (stylized model of transformers), by showing synchronization for all β ≥ -0.16, which significantly extends the previous bound of 0≤ β ≤ 1 from Criscitiello, Rebjock, McRae, and Boumal (2024). We also show that global synchronization does not occur when β < -2/3.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 16 of its 35 references that are in the map. First posted on arXiv 30 Jul 2025.

Builds on

16 of 35 references.

References · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023A mathematical perspective on Transformerscited by 164
  3. 2023The emergence of clusters in self-attention dynamicscited by 140
  4. 2021Sinkformers: Transformers with Doubly Stochastic Attentioncited by 138
  5. 2022Revisiting Over-smoothing in BERT from the Perspective of Graphcited by 72
  6. 2022Rank Diminishing in Deep Neural Networkscited by 47
  7. 2024Dynamic metastability in the self-attention modelcited by 46
  8. 2024Emergence of meta-stable clustering in mean-field transformer modelscited by 39
  9. 2024Clustering in Causal Attention Maskingcited by 32
  10. 2025A Unified Perspective on the Dynamics of Deep Transformerscited by 32
  11. 2025Quantitative Clustering in Mean-Field Transformer Modelscited by 26
  12. 2024Setting the Record Straight on Transformer Oversmoothingcited by 21
4 more
  1. 2024Synchronization on circles and spheres with nonlinear interactionscited by 17
  2. 2024The Asymptotic Behavior of Attention in Transformerscited by 16
  3. 2022Expander graphs are globally synchronizingcited by 8
  4. 2024Understanding Catastrophic Forgetting In LoRA via Mean-Field Attention Dynamicscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 9.

12 papers.

  1. 2025The Mean-Field Dynamics of Transformerscited by 30
  2. 2025A multiscale analysis of mean-field transformers in the moderate interaction regimecited by 25
  3. 2025Critical attention scaling in long-context transformerscited by 20
  4. 2026Perceptrons and localization of attention's mean-field landscapecited by 8
  5. 2026Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regimecited by 5
  6. 2026Random Quadratic Form on a Sphere: Synchronization by Common Noisecited by 4
  7. 2026On the Diverse Dynamical Behaviors Arising in Deep Linear Transformerscited by 3
  8. 2026YuriiFormer: A Suite of Nesterov-Accelerated Transformerscited by 2
  9. 2026Gradient Flow Structure and Quantitative Dynamics of Multi-Head Self-Attentioncited by 2
  10. 2026Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformerscited by 1
  11. 2026Kinetic theory for Transformers and the lost-in-the-middle phenomenoncited by 0
  12. 2026Asymptotic Behavior of Transformerscited by 0