Random Quadratic Form with random forcing: Metastable synchronization by noise
Not on the map. Why?
Abstract (arXiv)
We study the Random Quadratic Form (RQF) on a sphere in the presence of random Brownian forcing. We show that the forcing does not effectively change the law of the process but affects the synchronization properties of the system. While the RQF without forcing exhibits partial synchronization due to the intrinsic symmetries, the introduction of an arbitrarily small forcing results in long-term symmetry breaking and leads to full synchronization. In this work we focus on the small forcing regime and recover the multiscale behavior of the two-point process. We show that in the first stage the model converges to an anti-polar configuration due to the symmetries of the RQF and in the second stage the two clusters meet due to the symmetry breaking phenomenon. The model is motivated by continuous-time machine learning models such as Neural ODEs and continuous-time formulations of transformers. In particular, the results of this work explain the role of the bias and the scale of its initialization.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 47 references that are in the map. First posted on arXiv 17 Aug 2026.
Builds on
16 of 47 references.
References · 16
- 2017Attention Is All You Needcited by 59,932
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2023A mathematical perspective on Transformerscited by 160
- 2021Sinkformers: Transformers with Doubly Stochastic Attentioncited by 136
- 2023The emergence of clusters in self-attention dynamicscited by 133
- 2024Dynamic metastability in the self-attention modelcited by 42
- 2024Emergence of meta-stable clustering in mean-field transformer modelscited by 35
- 2025A multiscale analysis of mean-field transformers in the moderate interaction regimecited by 24
- 2025The Mean-Field Dynamics of Transformerscited by 24
- 2025Quantitative Clustering in Mean-Field Transformer Modelscited by 23
- 2025On the Structure of Stationary Solutions to McKean-Vlasov Equations with Applications to Noisy Transformerscited by 10
- 2026Clustering in Deep Stochastic Transformerscited by 5
4 more
- 2026Stochastic Scaling Limits and Synchronization by Noise in Deep Transformer Modelscited by 5
- 2026Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regimecited by 4
- 2026Random Quadratic Form on a Sphere: Synchronization by Common Noisecited by 2
- 2026On the Diverse Dynamical Behaviors Arising in Deep Linear Transformerscited by 2
Built on by
Not cited yet.