# Disentangling images with Lie group transformations and sparse coding

arXiv: https://arxiv.org/abs/2012.12071 · HTML: https://mlsanity.com/paper/2012.12071/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2020.

## Built on by (most cited first, 14 of 14)

- [Bispectral Neural Networks](https://mlsanity.com/paper/2209.03416/) (2022)
- [Neuromorphic Visual Scene Understanding with Resonator Networks](https://mlsanity.com/paper/2208.12880/) (2022)
- [A Generative Model of Symmetry Transformations](https://mlsanity.com/paper/2403.01946/) (2024)
- [A polar prediction model for learning to represent visual transformations](https://mlsanity.com/paper/2303.03432/) (2023)
- [Neural Isometries: Taming Transformations for Equivariant ML](https://mlsanity.com/paper/2405.19296/) (2024)
- [Learning and generalization of compositional representations of visual scenes](https://mlsanity.com/paper/2303.13691/) (2023)
- [Disentangling Patterns and Transformations from One Sequence of Images with Shape-invariant Lie Group Transformer](https://mlsanity.com/paper/2203.11210/) (2022)
- [Compositional Factorization of Visual Scenes with Convolutional Sparse Coding and Resonator Networks](https://mlsanity.com/paper/2404.19126/) (2024)
- [Recursive Neural Programs: Variational Learning of Image Grammars and Part-Whole Hierarchies](https://mlsanity.com/paper/2206.08462/) (2022)
- [A Differentiable Measure of Algebraic Complexity: Provably Exact Discovery of Group Structures](https://mlsanity.com/paper/2511.23152/) (2025)
- [Commutativity and Disentanglement from the Manifold Perspective](https://mlsanity.com/paper/2210.07857/) (2022)
- [MANGO: Learning Disentangled Image Transformation Manifolds with Grouped Operators](https://mlsanity.com/paper/2409.09542/) (2024)
- [Spectral Discovery of Continuous Symmetries via Generalized Fourier Transforms](https://mlsanity.com/paper/2603.07299/) (2026)
- [Commutator-Induced Uncertainty in VAEs](https://mlsanity.com/paper/2605.23449/) (2026)

## Abstract (arXiv)

Discrete spatial patterns and their continuous transformations are two important regularities contained in natural signals. Lie groups and representation theory are mathematical tools that have been used in previous works to model continuous image transformations. On the other hand, sparse coding is an important tool for learning dictionaries of patterns in natural signals. In this paper, we combine these ideas in a Bayesian generative model that learns to disentangle spatial patterns and their continuous transformations in a completely unsupervised manner. Images are modeled as a sparse superposition of shape components followed by a transformation that is parameterized by n continuous variables. The shape components and transformations are not predefined, but are instead adapted to learn the symmetries in the data, with the constraint that the transformations form a representation of an n-dimensional torus. Training the model on a dataset consisting of controlled geometric transformations of specific MNIST digits shows that it can recover these transformations along with the digits. Training on the full MNIST dataset shows that it can learn both the basic digit shapes and the natural transformations such as shearing and stretching that are contained in this data.