mlsanityData to 14 Oct 2026

Synchronization on circles and spheres with nonlinear interactions

arXiv 2405.18273 ↗First posted 28 May 2024math.OC cs.LG math.DScited by 17

Not on the map. Why?

Abstract (arXiv)

We consider the dynamics of n points on a sphere in ℝd (d ≥ 2) which attract each other according to a function varphi of their inner products. When varphi is linear (varphi(t) = t), the points converge to a common value (i.e., synchronize) in various connectivity scenarios: this is part of classical work on Kuramoto oscillator networks. When varphi is exponential (varphi(t) = eβ t), these dynamics correspond to a limit of how idealized transformers process data, as described by Geshkovski et al. (2025). Accordingly, they ask whether synchronization occurs for exponential varphi. The answer depends on the dimension d. In the context of consensus for multi-agent control, Markdahl et al. (2018) show that for d ≥ 3 (spheres), if the interaction graph is connected and varphi is increasing and convex, then the system synchronizes. We give a separate proof of this result. What is the situation on circles (d=2)? First, we show that varphi being increasing and convex is no longer sufficient (even for complete graphs). Then we identify a new condition under which we do have synchronization on the circle (namely, if the Taylor coefficients of varphi' are decreasing). As a corollary, this provide synchronization for exponential varphi with β ∈ (0, 1]. The proofs are based on nonconvex landscape analysis.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 14 of its 57 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

14 of 57 references.

References · 14

  1. 2018Neural Ordinary Differential Equationscited by 3,779
  2. 2017Stable Architectures for Deep Neural Networkscited by 341
  3. 2023A mathematical perspective on Transformerscited by 164
  4. 2021Sinkformers: Transformers with Doubly Stochastic Attentioncited by 138
  5. 2013Manopt, a Matlab toolbox for optimization on manifoldscited by 113
  6. 2019Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of Viewcited by 113
  7. 2021Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsitycited by 85
  8. 2023Saddle-to-Saddle Dynamics in Diagonal Linear Networkscited by 57
  9. 2021Redesigning the Transformer Architecture with Insights from Multi-particle Dynamical Systemscited by 31
  10. 2022Incremental Learning in Diagonal Linear Networkscited by 28
  11. 2018On the Landscape of Synchronization Networks: A Perspective from Nonconvex Optimizationcited by 11
  12. 2022Expander graphs are globally synchronizingcited by 8
2 more
  1. 2023Benign landscapes of low-dimensional relaxations for orthogonal synchronization on general graphscited by 6
  2. 2019Synchronization on Riemannian manifolds: Multiply connected implies multistablecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 8.

17 papers.

Sort: