mlsanityData to 14 Oct 2026

Identifiable Equivariant Networks are Layerwise Equivariant

arXiv 2601.21645 ↗First posted 29 Jan 2026cs.LG math.CT math.RTcited by 7

Not on the map. Why?

Abstract (arXiv)

We investigate the relation between end-to-end equivariance and layerwise equivariance in deep neural networks. We prove the following: For a network whose end-to-end function is equivariant with respect to group actions on the input and output spaces, there is a parameter choice yielding the same end-to-end function such that its layers are equivariant with respect to some group actions on the latent spaces. Our result assumes that the parameters of the model are identifiable in an appropriate sense. This identifiability property has been established in the literature for a large class of networks, to which our results apply immediately, while it is conjectural for others. The theory we develop is grounded in an abstract formalism, and is therefore architecture-agnostic. Overall, our results provide a mathematical explanation for the emergence of equivariant structures in the weights of neural networks during training -- a phenomenon that is consistently observed in practice.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

34 of 43 references.

References · 34

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  3. 2018Invariant and Equivariant Graph Networkscited by 346
  4. 2014Understanding image representations by measuring their equivariance and equivalencecited by 243
  5. 2019On the Universality of Invariant Networkscited by 152
  6. 2018Universal approximations of invariant maps by neural networkscited by 127
  7. 2020On the Universality of Rotation Equivariant Point Cloud Networkscited by 80
  8. 2019On the Expressive Power of Deep Polynomial Neural Networkscited by 66
  9. 2022On the Symmetries of Deep Learning Models and their Internal Representationscited by 64
  10. 2022The Lie Derivative for Measuring Learned Equivariancecited by 54
  11. 2024Does equivariance matter at scale?cited by 52
  12. 2020Universal Equivariant Multilayer Perceptronscited by 49
22 more
  1. 2017Backprop as Functor: A compositional perspective on supervised learningcited by 44
  2. 2023Hidden symmetries of ReLU networkscited by 39
  3. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  4. 2025Symmetry in Neural Network Parameter Spacescited by 36
  5. 2019Pure and Spurious Critical Points: a Geometric Study of Linear Networkscited by 33
  6. 2024Position: Categorical Deep Learning is an Algebraic Theory of All Architecturescited by 32
  7. 2023Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networkscited by 23
  8. 2024The Empirical Impact of Neural Parameter Symmetries, or Lack Thereofcited by 21
  9. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  10. 2020Affine symmetries and neural network identifiabilitycited by 18
  11. 2023Optimization Dynamics of Equivariant and Augmented Neural Networkscited by 18
  12. 2024Geometry of Lightning Self-Attention: Identifiability and Dimensioncited by 16
  13. 2019Neural network identifiability for a family of sigmoidal nonlinearitiescited by 14
  14. 2024Activation degree thresholds and expressiveness of polynomial neural networkscited by 14
  15. 2022Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groupscited by 11
  16. 2024On the Geometry and Optimization of Polynomial Convolutional Networkscited by 11
  17. 2023Investigating how ReLU-networks encode symmetriescited by 8
  18. 2025A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Modelscited by 8
  19. 2025The Alexander-Hirschowitz theorem for neurovarietiescited by 6
  20. 2022A Classification of G-invariant Shallow Neural Networkscited by 5
  21. 2025Universally Invariant Learning in Equivariant GNNscited by 3
  22. 2023Densely Connected G-invariant Deep Neural Networks with Signed Permutation Representationscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Most ReLU Networks Admit Identifiable Parameterscited by 6
  2. 2026The Geometry of Polynomial Group Convolutional Neural Networkscited by 2
  3. 2026Conservation Laws from Data Symmetry in Neural Networkscited by 1
  4. 2026Singular Learning and Occam's Razor in Deep Monomial Networkscited by 0
  5. 2026Boosting Data Augmentation with Stochastic Weight Averagingcited by 0
  6. 2026Linear Independence of Polynomial Compositions and Identifiability of Deep Neural Networkscited by 0
  7. 2026Inherent Trade-offs when Enforcing Correct Data Symmetries in Multi-head Self-attentioncited by 0