Conservation Laws from Data Symmetry in Neural Networks
Not on the map. Why?
Abstract (arXiv)
We explore whether intrinsic symmetries of the training data lead to conserved quantities during gradient-flow training of neural networks. Under the assumption that the loss function is analytic and non-polynomial, we prove that data symmetries generically do not induce any additional integrals of motion. For mean squared error (MSE) loss, on the other hand, there are situations in which data augmentation yields extra conserved quantities. We build a framework, utilizing tensorizable networks to describe this phenomenon. Tensorizable networks are a family of architectures whose dependence on parameters and inputs can be separated using an intermediate representation. They include linear and polynomial networks, as well as Lightning Attention.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 38 references that are in the map. First posted on arXiv 9 Jun 2026.
Builds on
28 of 38 references.
References · 28
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
- 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
- 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
- 2019Implicit Regularization in Deep Matrix Factorizationcited by 402
- 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
- 2018Characterizing Implicit Bias in Terms of Optimization Geometrycited by 259
- 2018Gradient descent aligns the layers of deep linear networkscited by 182
- 2018A Convergence Analysis of Gradient Descent for Deep Linear Neural Networkscited by 180
- 2018Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balancedcited by 172
- 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
- 2020Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamicscited by 93
16 more
- 2021Learning with invariances in random features and kernel modelscited by 89
- 2020On the Benefits of Invariance in Neural Networkscited by 78
- 2021Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networkscited by 47
- 2023Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flowscited by 44
- 2023Hidden symmetries of ReLU networkscited by 38
- 2022Symmetries, flat minima, and the conserved quantities of gradient flowcited by 35
- 2021Noether: The More Things Change, the More Stay the Samecited by 19
- 2023Optimization Dynamics of Equivariant and Augmented Neural Networkscited by 17
- 2019Neural network identifiability for a family of sigmoidal nonlinearitiescited by 14
- 2025Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networkscited by 9
- 2023On the Implicit Bias of Linear Equivariant Steerable Networkscited by 7
- 2026Identifiable Equivariant Networks are Layerwise Equivariantcited by 7
- 2026Most ReLU Networks Admit Identifiable Parameterscited by 5
- 2025Data Augmentation and Regularization for Learning Group Equivariancecited by 3
- 2025Understanding Learning Invariance in Deep Linear Networkscited by 3
- 2026The Geometry of Polynomial Group Convolutional Neural Networkscited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.