# Disentangling Patterns and Transformations from One Sequence of Images with Shape-invariant Lie Group Transformer

arXiv: https://arxiv.org/abs/2203.11210 · HTML: https://mlsanity.com/paper/2203.11210/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 6 of 6)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [Disentangling images with Lie group transformations and sparse coding](https://mlsanity.com/paper/2012.12071/) (2020)

## Built on by (most cited first, 3 of 3)

- [Learning Conditionally Independent Transformations using Normal Subgroups in Group Theory](https://mlsanity.com/paper/2504.04490/) (2025)
- [Feature-Based Lie Group Transformer for Real-World Applications](https://mlsanity.com/paper/2506.04668/) (2025)
- [Transformation Categorization Based on Group Decomposition Theory Using Parameter Division](https://mlsanity.com/paper/2605.04056/) (2026)

## Abstract (arXiv)

An effective way to model the complex real world is to view the world as a composition of basic components of objects and transformations. Although humans through development understand the compositionality of the real world, it is extremely difficult to equip robots with such a learning mechanism. In recent years, there has been significant research on autonomously learning representations of the world using the deep learning; however, most studies have taken a statistical approach, which requires a large number of training data. Contrary to such existing methods, we take a novel algebraic approach for representation learning based on a simpler and more intuitive formulation that the observed world is the combination of multiple independent patterns and transformations that are invariant to the shape of patterns. Since the shape of patterns can be viewed as the invariant features against symmetric transformations such as translation or rotation, we can expect that the patterns can naturally be extracted by expressing transformations with symmetric Lie group transformers and attempting to reconstruct the scene with them. Based on this idea, we propose a model that disentangles the scenes into the minimum number of basic components of patterns and Lie transformations from only one sequence of images, by introducing the learnable shape-invariant Lie group transformers as transformation components. Experiments show that given one sequence of images in which two objects are moving independently, the proposed model can discover the hidden distinct objects and multiple shape-invariant transformations that constitute the scenes.