# LieGG: Studying Learned Lie Group Generators

arXiv: https://arxiv.org/abs/2210.04345 · HTML: https://mlsanity.com/paper/2210.04345/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 53 references that are in the map.
- First posted on arXiv 9 Oct 2022.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Rethinking ImageNet Pre-training](https://mlsanity.com/paper/1811.08883/) (2018)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Rethinking Pre-training and Self-training](https://mlsanity.com/paper/2006.06882/) (2020)
- [Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers](https://mlsanity.com/paper/1811.04918/) (2018)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth](https://mlsanity.com/paper/2010.15327/) (2020)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [Deep Scale-spaces: Equivariance Over Scale](https://mlsanity.com/paper/1905.11697/) (2019)
- [Scale-Equivariant Steerable Networks](https://mlsanity.com/paper/1910.11093/) (2019)
- [B-Spline CNNs on Lie Groups](https://mlsanity.com/paper/1909.12057/) (2019)
- [Approximately Equivariant Networks for Imperfectly Symmetric Dynamics](https://mlsanity.com/paper/2201.11969/) (2022)
- [Automatic Symmetry Discovery with Lie Algebra Convolutional Network](https://mlsanity.com/paper/2109.07103/) (2021)
- [Meta-Learning Symmetries by Reparameterization](https://mlsanity.com/paper/2007.02933/) (2020)
- [Residual Pathway Priors for Soft Equivariance Constraints](https://mlsanity.com/paper/2112.01388/) (2021)
- [The Lie Derivative for Measuring Learned Equivariance](https://mlsanity.com/paper/2210.02984/) (2022)
- [DISCO: accurate Discrete Scale Convolutions](https://mlsanity.com/paper/2106.02733/) (2021)
- [Scale Equivariance Improves Siamese Tracking](https://mlsanity.com/paper/2007.09115/) (2020)

## Built on by (most cited first, 30 of 31)

- [Generative Adversarial Symmetry Discovery](https://mlsanity.com/paper/2302.00236/) (2023)
- [Latent Space Symmetry Discovery](https://mlsanity.com/paper/2310.00105/) (2023)
- [Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principles](https://mlsanity.com/paper/2301.05638/) (2023)
- [A General Theory of Correct, Incorrect, and Extrinsic Equivariance](https://mlsanity.com/paper/2303.04745/) (2023)
- [What Affects Learned Equivariance in Deep Image Recognition Models?](https://mlsanity.com/paper/2304.02628/) (2023)
- [On genuine invariance learning without weight-tying](https://mlsanity.com/paper/2308.03904/) (2023)
- [Discovering Sparse Representations of Lie Groups with Machine Learning](https://mlsanity.com/paper/2302.05383/) (2023)
- [Symmetry Discovery for Different Data Types](https://mlsanity.com/paper/2410.09841/) (2024)
- [Color Equivariant Convolutional Networks](https://mlsanity.com/paper/2310.19368/) (2023)
- [Accelerated Discovery of Machine-Learned Symmetries: Deriving the Exceptional Lie Groups G2, F4 and E6](https://mlsanity.com/paper/2307.04891/) (2023)
- [Identifying the Group-Theoretic Structure of Machine-Learned Symmetries](https://mlsanity.com/paper/2309.07860/) (2023)
- [Neural Isometries: Taming Transformations for Equivariant ML](https://mlsanity.com/paper/2405.19296/) (2024)
- [Oracle-Preserving Latent Flows](https://mlsanity.com/paper/2302.00806/) (2023)
- [Equivariant Graph Neural Networks for Charged Particle Tracking](https://mlsanity.com/paper/2304.05293/) (2023)
- [Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensors](https://mlsanity.com/paper/2412.04594/) (2024)
- [Governing Equation Discovery from Data Based on Differential Invariants](https://mlsanity.com/paper/2505.18798/) (2025)
- [Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learning](https://mlsanity.com/paper/2401.08603/) (2023)
- [Learning Equivariant Functions via Quadratic Forms](https://mlsanity.com/paper/2509.22184/) (2025)
- [Manifold Contrastive Learning with Variational Lie Group Operators](https://mlsanity.com/paper/2306.13544/) (2023)
- [Stationary Velocity Fields on Matrix Groups for Deformable Image Registration](https://mlsanity.com/paper/2410.10997/) (2024)
- [Learning Linear Groups in Neural Networks](https://mlsanity.com/paper/2305.18552/) (2023)
- [Compositional Symmetry as Compression: Lie Pseudogroup Structure in Algorithmic Agents](https://mlsanity.com/paper/2510.10586/) (2025)
- [Spectral Discovery of Continuous Symmetries via Generalized Fourier Transforms](https://mlsanity.com/paper/2603.07299/) (2026)
- [Blind Recovery of Latent Domains via Unsupervised Symmetry Discovery](https://mlsanity.com/paper/2606.17782/) (2026)
- [Observable- and Positional-Encoding-Dependent Symmetry Readout from Neural Network Weights](https://mlsanity.com/paper/2607.03108/) (2026)
- [On the post-hoc Evaluation of PDE Discovery: A Multifaceted Challenge of Scientific Advancement](https://mlsanity.com/paper/2607.23753/) (2026)
- [Discovering Symmetries in Neural Network Parameter Spaces](https://mlsanity.com/paper/2609.33527/) (2026)
- [LieDiscover: Adaptive Symbolic Library Construction for Explicit Open-form Symmetry Discovery](https://mlsanity.com/paper/2609.33663/) (2026)
- [Symmetry Discovery in Quantum Learning: Observable-Level and Task-Level Inference from Finite Measurements](https://mlsanity.com/paper/2610.00157/) (2026)
- [Learning Operators for Symmetry Discovery: From Lie Groups to Local Symmetry Geometry](https://mlsanity.com/paper/2610.16884/) (2026)

## Abstract (arXiv)

Symmetries built into a neural network have appeared to be very beneficial for a wide range of tasks as it saves the data to learn them. We depart from the position that when symmetries are not built into a model a priori, it is advantageous for robust networks to learn symmetries directly from the data to fit a task function. In this paper, we present a method to extract symmetries learned by a neural network and to evaluate the degree to which a network is invariant to them. With our method, we are able to explicitly retrieve learned invariances in a form of the generators of corresponding Lie-groups without prior knowledge of symmetries in the data. We use the proposed method to study how symmetrical properties depend on a neural network's parameterization and configuration. We found that the ability of a network to learn symmetries generalizes over a range of architectures. However, the quality of learned symmetries depends on the depth and the number of parameters.