mlsanityData to 7 Oct 2026

LieGG: Studying Learned Lie Group Generators

arXiv 2210.04345 ↗First posted 9 Oct 2022cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Symmetries built into a neural network have appeared to be very beneficial for a wide range of tasks as it saves the data to learn them. We depart from the position that when symmetries are not built into a model a priori, it is advantageous for robust networks to learn symmetries directly from the data to fit a task function. In this paper, we present a method to extract symmetries learned by a neural network and to evaluate the degree to which a network is invariant to them. With our method, we are able to explicitly retrieve learned invariances in a form of the generators of corresponding Lie-groups without prior knowledge of symmetries in the data. We use the proposed method to study how symmetrical properties depend on a neural network's parameterization and configuration. We found that the ability of a network to learn symmetries generalizes over a range of architectures. However, the quality of learned symmetries depends on the depth and the number of parameters.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 9 Oct 2022.

Builds on

35 of 53 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  5. 2017Deep Setscited by 1,394
  6. 2016Group Equivariant Convolutional Networkscited by 1,154
  7. 2016Error bounds for approximations with deep ReLU networkscited by 567
  8. 2019General E(2)-Equivariant Steerable CNNscited by 441
  9. 2018Rethinking ImageNet Pre-trainingcited by 376
  10. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  11. 2020Rethinking Pre-training and Self-trainingcited by 352
  12. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
23 more
  1. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  2. 2016On the Expressive Power of Deep Neural Networkscited by 302
  3. 2019On the Relationship between Self-Attention and Convolutional Layerscited by 299
  4. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
  5. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
  6. 2014Understanding image representations by measuring their equivariance and equivalencecited by 240
  7. 2019Universal Approximation with Deep Narrow Networkscited by 198
  8. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
  9. 2019Deep Scale-spaces: Equivariance Over Scalecited by 113
  10. 2019Scale-Equivariant Steerable Networkscited by 110
  11. 2019B-Spline CNNs on Lie Groupscited by 97
  12. 2022Approximately Equivariant Networks for Imperfectly Symmetric Dynamicscited by 95
  13. 2021Automatic Symmetry Discovery with Lie Algebra Convolutional Networkcited by 82
  14. 2020Meta-Learning Symmetries by Reparameterizationcited by 75
  15. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 74
  16. 2022The Lie Derivative for Measuring Learned Equivariancecited by 51
  17. 2021DISCO: accurate Discrete Scale Convolutionscited by 35
  18. 2020Scale Equivariance Improves Siamese Trackingcited by 32
  19. 2021Steerable Partial Differential Operators for Equivariant Neural Networkscited by 22
  20. 2021Uniform Generalization Bounds for Overparameterized Neural Networkscited by 21
  21. 2019Representing Closed Transformation Paths in Encoded Network Latent Spacecited by 17
  22. 2021Implicit Equivariance in Convolutional Networkscited by 5
  23. 2017Dynamic Steerable Blocks in Deep Residual Networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 4; 2025: 3; 2026: 6.

28 papers.

Sort:

  1. 2023Generative Adversarial Symmetry Discoverycited by 40
  2. 2023Latent Space Symmetry Discoverycited by 29
  3. 2023Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principlescited by 22
  4. 2023A General Theory of Correct, Incorrect, and Extrinsic Equivariancecited by 18
  5. 2023What Affects Learned Equivariance in Deep Image Recognition Models?cited by 14
  6. 2023On genuine invariance learning without weight-tyingcited by 13
  7. 2023Discovering Sparse Representations of Lie Groups with Machine Learningcited by 8
  8. 2023Color Equivariant Convolutional Networkscited by 7
  9. 2024Symmetry Discovery for Different Data Typescited by 7
  10. 2023Accelerated Discovery of Machine-Learned Symmetries: Deriving the Exceptional Lie Groups G2, F4 and E6cited by 6
  11. 2023Identifying the Group-Theoretic Structure of Machine-Learned Symmetriescited by 6
  12. 2024Neural Isometries: Taming Transformations for Equivariant MLcited by 6
16 more
  1. 2023Oracle-Preserving Latent Flowscited by 5
  2. 2023Equivariant Graph Neural Networks for Charged Particle Trackingcited by 5
  3. 2024Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensorscited by 5
  4. 2025Governing Equation Discovery from Data Based on Differential Invariantscited by 4
  5. 2023Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learningcited by 2
  6. 2025Learning Equivariant Functions via Quadratic Formscited by 2
  7. 2023Manifold Contrastive Learning with Variational Lie Group Operatorscited by 1
  8. 2024Stationary Velocity Fields on Matrix Groups for Deformable Image Registrationcited by 1
  9. 2023Learning Linear Groups in Neural Networkscited by 0
  10. 2025Compositional Symmetry as Compression: Lie Pseudogroup Structure in Algorithmic Agentscited by 0
  11. 2026Spectral Discovery of Continuous Symmetries via Generalized Fourier Transformscited by 0
  12. 2026Blind Recovery of Latent Domains via Unsupervised Symmetry Discoverycited by 0
  13. 2026Observable- and Positional-Encoding-Dependent Symmetry Readout from Neural Network Weightscited by 0
  14. 2026On the post-hoc Evaluation of PDE Discovery: A Multifaceted Challenge of Scientific Advancementcited by 0
  15. 2026Discovering Symmetries in Neural Network Parameter Spacescited by 0
  16. 2026LieDiscover: Adaptive Symbolic Library Construction for Explicit Open-form Symmetry Discoverycited by 0