mlsanityData to 14 Oct 2026

Spontaneous symmetry breaking and Goldstone modes for deep information propagation

arXiv 2605.14685 ↗First posted 14 May 2026cs.LG cond-mat.stat-mech cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation over long distances and times. In this work, we study deep neural networks whose internal layers are equivariant under a continuous symmetry and may therefore support analogous Goldstone-like degrees of freedom. We demonstrate, both analytically and empirically, that these degrees of freedom enable coherent signal propagation across depth and recurrent iterations, providing a mechanism for stable information flow without relying on architectural stabilizers such as residual connections or normalization. In feedforward networks, this results in improved trainability and representational diversity across layers. In recurrent settings, we demonstrate the same mechanism is valuable for long-term memory by propagating information over recurrent iterations, thereby improving performance of RNNs and GRUs on long-sequence modeling tasks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 33 of its 61 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

33 of 61 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  5. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  6. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  7. 2016Group Equivariant Convolutional Networkscited by 1,165
  8. 2017Dynamic Routing Between Capsulescited by 689
  9. 2014Neural Turing Machinescited by 644
  10. 2017Deep Neural Networks as Gaussian Processescited by 622
  11. 2015Unitary Evolution Recurrent Neural Networkscited by 294
  12. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
21 more
  1. 2016Deep Information Propagationcited by 195
  2. 2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
  3. 2015Delving Deeper into Convolutional Networks for Learning Video Representationscited by 148
  4. 2018Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNNcited by 143
  5. 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
  6. 2020Lipschitz Recurrent Neural Networkscited by 89
  7. 2017Mean Field Residual Networks: On the Edge of Chaoscited by 83
  8. 2020Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependenciescited by 82
  9. 2018Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networkscited by 45
  10. 2021Long Expressive Memory for Sequence Modelingcited by 41
  11. 2022Complex-Valued Autoencoders for Object Discoverycited by 40
  12. 2016Recurrent Orthogonal Networks and Long-Memory Taskscited by 38
  13. 2024Artificial Kuramoto Oscillatory Neuronscited by 38
  14. 2020Finding Symmetry Breaking Order Parameters with Euclidean Neural Networkscited by 30
  15. 2023Symmetry Breaking and Equivariant Neural Networkscited by 25
  16. 2023Traveling Waves Encode the Recent Past and Enhance Sequence Learningcited by 16
  17. 2017A Correspondence Between Random Neural Networks and Statistical Field Theorycited by 14
  18. 2023Image segmentation with traveling waves in an exactly solvable recurrent neural networkcited by 8
  19. 2024Binding Dynamics in Rotating Featurescited by 5
  20. 2026Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priorscited by 2
  21. 2026Symmetry, Defects, and Diffusion in Continuous-memory Recurrent Networkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Correlation flow governs learning at criticalitycited by 0
  2. 2026Deep neural networks as lattice gauge theoriescited by 0