mlsanityData to 7 Oct 2026

Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlators

arXiv 2106.00694 ↗First posted 1 Jun 2021cs.LG hep-th stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

Parameter-space and function-space provide two different duality frames in which to study neural networks. We demonstrate that symmetries of network densities may be determined via dual computations of network correlation functions, even when the density is unknown and the network is not equivariant. Symmetry-via-duality relies on invariance properties of the correlation functions, which stem from the choice of network parameter distributions. Input and output symmetries of neural network densities are determined, which recover known Gaussian process results in the infinite width limit. The mechanism may also be utilized to determine symmetries during training, when parameters are correlated, as well as symmetries of the Neural Tangent Kernel. We demonstrate that the amount of symmetry in the initialization density affects the accuracy of networks trained on Fashion-MNIST, and that symmetry breaking helps only when it is in the direction of ground truth.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 39 of its 53 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

39 of 53 references.

References · 39

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2016Group Equivariant Convolutional Networkscited by 1,154
  4. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  5. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  6. 2017Deep Neural Networks as Gaussian Processescited by 617
  7. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  8. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  9. 2018Invariant and Equivariant Graph Networkscited by 339
  10. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  11. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 296
  12. 2018Towards a Definition of Disentangled Representationscited by 294
27 more
  1. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
  2. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 262
  3. 2019Cormorant: Covariant Molecular Neural Networkscited by 262
  4. 2020Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densitiescited by 248
  5. 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 168
  6. 2019On the Universality of Invariant Networkscited by 151
  7. 2019Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processescited by 148
  8. 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 146
  9. 2017Equivariance Through Parameter-Sharingcited by 139
  10. 2018Deep Convolutional Networks as shallow Gaussian Processescited by 137
  11. 2020Lorentz Group Equivariant Neural Network for Particle Physicscited by 120
  12. 2020Equivariant flow-based sampling for lattice gauge theorycited by 102
  13. 2019B-Spline CNNs on Lie Groupscited by 97
  14. 2020On Learning Sets of Symmetric Elementscited by 97
  15. 2020Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamicscited by 93
  16. 2020Sampling using SU(N) gauge equivariant flowscited by 83
  17. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  18. 2019Asymptotics of Wide Networks from Feynman Diagramscited by 77
  19. 2018Explorations in Homeomorphic Variational Auto-Encodingcited by 65
  20. 2020Neural Networks and Quantum Field Theorycited by 58
  21. 2020Detecting Symmetries with Neural Networkscited by 33
  22. 2018Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)cited by 16
  23. 2019Learning Curves for Deep Neural Networks: A Gaussian Field Theory Perspectivecited by 11
  24. 2020A Data and Compute Efficient Design for Limited-Resources Deep Learningcited by 10
  25. 2020Connecting Dualities and Machine Learningcited by 5
  26. 2019Set Flow: A Permutation Invariant Normalizing Flowcited by 4
  27. 2019Exchangeable Generative Models with Flow Scanscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 4; 2023: 3; 2024: 6; 2025: 8; 2026: 4.

28 papers.

Sort:

  1. 2021Nonperturbative renormalization for the neural network-QFT correspondencecited by 24
  2. 2021The edge of chaos: quantum field theory and deep neural networkscited by 23
  3. 2023Neural Network Field Theories: Non-Gaussianity, Actions, and Localitycited by 21
  4. 2023Structures of Neural Network Effective Theoriescited by 13
  5. 2024Rigor with Machine Learning from Field Theory to the Poincar'e Conjecturecited by 12
  6. 2022A duality connecting neural network and cosmological dynamicscited by 10
  7. 2024Conformal Fields from Neural Networkscited by 10
  8. 2024Bayesian RG Flow in Neural Network Field Theoriescited by 9
  9. 2022Renormalization in the neural network-quantum field theory correspondencecited by 7
  10. 2024NCoder -- A Quantum Field Theory approach to encoding datacited by 7
  11. 2025Fermions and Supersymmetry in Neural Network Field Theoriescited by 5
  12. 2021Infinite Neural Network Quantum States: Entanglement and Training Dynamicscited by 4
16 more
  1. 2024TASI Lectures on Physics for Machine Learningcited by 4
  2. 2022Data Science and Machine Learning in Educationcited by 3
  3. 2025Finite-Width Neural Tangent Kernels from Feynman Diagramscited by 3
  4. 2022p-Adic Statistical Field Theory and Deep Belief Networkscited by 2
  5. 2024Generative AI for Brane Configurations and Coamoebacited by 2
  6. 2025The neural networks with tensor weights and emergent fermionic Wick rules in the large-width limitcited by 2
  7. 2025Gauge-covariant stochastic neural fields: Stability and finite-width effectscited by 2
  8. 2025Conformal Defects in Neural Network Field Theoriescited by 2
  9. 2025Virasoro Symmetry in Neural Network Field Theoriescited by 2
  10. 2026Topological Effects in Neural Network Field Theorycited by 1
  11. 2023Black holes and the loss landscape in machine learningcited by 0
  12. 2025Bulk-boundary decomposition of neural networkscited by 0
  13. 2025Quantitative Understanding of PDF Fits and their Uncertaintiescited by 0
  14. 2026What Neural Network Field Theory Can and Cannot Realise on a Computercited by 0
  15. 2026Parameter-Level Attribution of Symmetry in Trained Networks Though Parameter-Wise Functional Sensitivitycited by 0
  16. 2026Eigenvalues of the Hessian in Deep Learning: The Origin of Symmetry and Its Breakingcited by 0