# Addressing the Topological Defects of Disentanglement via Distributed Operators

arXiv: https://arxiv.org/abs/2102.05623 · HTML: https://mlsanity.com/paper/2102.05623/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 10 Feb 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [Flexibly Fair Representation Learning by Disentanglement](https://mlsanity.com/paper/1906.02589/) (2019)
- [Hyperspherical Variational Auto-Encoders](https://mlsanity.com/paper/1804.00891/) (2018)
- [On the Fairness of Disentangled Representations](https://mlsanity.com/paper/1905.13662/) (2019)
- [Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects](https://mlsanity.com/paper/1811.11553/) (2018)
- [Weakly Supervised Disentanglement with Guarantees](https://mlsanity.com/paper/1910.09772/) (2019)
- [Polar Transformer Networks](https://mlsanity.com/paper/1709.01889/) (2017)
- [Meta-Learning Symmetries by Reparameterization](https://mlsanity.com/paper/2007.02933/) (2020)
- [Explorations in Homeomorphic Variational Auto-Encoding](https://mlsanity.com/paper/1807.04689/) (2018)
- [Attentive Group Equivariant Convolutional Networks](https://mlsanity.com/paper/2002.03830/) (2020)
- [Learning Invariances in Neural Networks](https://mlsanity.com/paper/2010.11882/) (2020)
- [Equivariant Neural Rendering](https://mlsanity.com/paper/2006.07630/) (2020)
- [Learning the Irreducible Representations of Commutative Lie Groups](https://mlsanity.com/paper/1402.4437/) (2014)
- [Variational Autoencoder with Learned Latent Structure](https://mlsanity.com/paper/2006.10597/) (2020)
- [Equivariant Transformer Networks](https://mlsanity.com/paper/1901.11399/) (2019)
- [Symmetry-Based Disentangled Representation Learning requires Interaction with Environments](https://mlsanity.com/paper/1904.00243/) (2019)
- [Only Bayes should learn a manifold (on the estimation of differential geometric structure from data)](https://mlsanity.com/paper/1806.04994/) (2018)
- [Representing Closed Transformation Paths in Encoded Network Latent Space](https://mlsanity.com/paper/1912.02644/) (2019)
- [Learning Group Structure and Disentangled Representations of Dynamical Environments](https://mlsanity.com/paper/2002.06991/) (2020)
- [Unsupervised Transformation Learning via Convex Relaxations](https://mlsanity.com/paper/1711.02226/) (2017)
- [No Representation without Transformation](https://mlsanity.com/paper/1912.03845/) (2019)

## Built on by (most cited first, 14 of 14)

- [Cross-Entropy Is All You Need To Invert the Data Generating Process](https://mlsanity.com/paper/2410.21869/) (2024)
- [Towards Building A Group-based Unsupervised Representation Disentanglement Framework](https://mlsanity.com/paper/2102.10303/) (2021)
- [Unsupervised Learning of Equivariant Structure from Sequences](https://mlsanity.com/paper/2210.05972/) (2022)
- [DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation](https://mlsanity.com/paper/2107.12549/) (2021)
- [Flow Factorized Representation Learning](https://mlsanity.com/paper/2309.13167/) (2023)
- [A Generative Model of Symmetry Transformations](https://mlsanity.com/paper/2403.01946/) (2024)
- [On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory](https://mlsanity.com/paper/2412.11521/) (2024)
- [Robust Self-Supervised Learning with Lie Groups](https://mlsanity.com/paper/2210.13356/) (2022)
- [CFASL: Composite Factor-Aligned Symmetry Learning for Disentanglement in Variational AutoEncoder](https://mlsanity.com/paper/2401.08897/) (2024)
- [Manifold Contrastive Learning with Variational Lie Group Operators](https://mlsanity.com/paper/2306.13544/) (2023)
- [Topographic VAEs learn Equivariant Capsules](https://mlsanity.com/paper/2109.01394/) (2021)
- [Invariance-adapted decomposition and Lasso-type contrastive learning](https://mlsanity.com/paper/2210.07413/) (2022)
- [Disentanglement and Compositionality of Letter Identity and Letter Position in Variational Auto-Encoder Vision Models](https://mlsanity.com/paper/2412.10446/) (2024)
- [Latent Equivariant Operators for Robust Object Recognition: Promises and Challenges](https://mlsanity.com/paper/2602.18406/) (2026)

## Abstract (arXiv)

A core challenge in Machine Learning is to learn to disentangle natural factors of variation in data (e.g. object shape vs. pose). A popular approach to disentanglement consists in learning to map each of these factors to distinct subspaces of a model's latent representation. However, this approach has shown limited empirical success to date. Here, we show that, for a broad family of transformations acting on images--encompassing simple affine transformations such as rotations and translations--this approach to disentanglement introduces topological defects (i.e. discontinuities in the encoder). Motivated by classical results from group representation theory, we study an alternative, more flexible approach to disentanglement which relies on distributed latent operators, potentially acting on the entire latent space. We theoretically and empirically demonstrate the effectiveness of this approach to disentangle affine transformations. Our work lays a theoretical foundation for the recent success of a new generation of models using distributed operators for disentanglement.