# Topological Obstructions and How to Avoid Them

arXiv: https://arxiv.org/abs/2312.07529 · HTML: https://mlsanity.com/paper/2312.07529/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 30 of 30)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Hyperspherical Variational Auto-Encoders](https://mlsanity.com/paper/1804.00891/) (2018)
- [Riemannian Continuous Normalizing Flows](https://mlsanity.com/paper/2006.10605/) (2020)
- [Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group](https://mlsanity.com/paper/1901.08428/) (2019)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Normalizing Flows on Tori and Spheres](https://mlsanity.com/paper/2002.02428/) (2020)
- [Spherical Latent Spaces for Stable Variational Autoencoders](https://mlsanity.com/paper/1808.10805/) (2018)
- [Explorations in Homeomorphic Variational Auto-Encoding](https://mlsanity.com/paper/1807.04689/) (2018)
- [Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold](https://mlsanity.com/paper/2106.05965/) (2021)
- [Learning Symmetric Embeddings for Equivariant World Models](https://mlsanity.com/paper/2204.11371/) (2022)
- [Spherical Text Embedding](https://mlsanity.com/paper/1911.01196/) (2019)
- [Equivariant Neural Rendering](https://mlsanity.com/paper/2006.07630/) (2020)
- [Unsupervised Learning of Group Invariant and Equivariant Representations](https://mlsanity.com/paper/2202.07559/) (2022)
- [Symmetry-Based Disentangled Representation Learning requires Interaction with Environments](https://mlsanity.com/paper/1904.00243/) (2019)
- [Disentangling by Subspace Diffusion](https://mlsanity.com/paper/2006.12982/) (2020)
- [Commutative Lie Group VAE for Disentanglement Learning](https://mlsanity.com/paper/2106.03375/) (2021)
- [Rigid Body Flows for Sampling Molecular Crystal Structures](https://mlsanity.com/paper/2301.11355/) (2023)
- [Symmetry-Based Representations for Artificial and Biological General Intelligence](https://mlsanity.com/paper/2203.09250/) (2022)
- [Towards Building A Group-based Unsupervised Representation Disentanglement Framework](https://mlsanity.com/paper/2102.10303/) (2021)
- [Topological Obstructions to Autoencoding](https://mlsanity.com/paper/2102.08380/) (2021)
- [Equivariant Representation Learning via Class-Pose Decomposition](https://mlsanity.com/paper/2207.03116/) (2022)
- [Delving into Discrete Normalizing Flows on SO(3) Manifold for Probabilistic Rotation Modeling](https://mlsanity.com/paper/2304.03937/) (2023)
- [On Incorporating Inductive Biases into VAEs](https://mlsanity.com/paper/2106.13746/) (2021)
- [Quantifying and Learning Linear Symmetry-Based Disentanglement](https://mlsanity.com/paper/2011.06070/) (2020)
- [Topological Constraints on Homeomorphic Auto-Encoding](https://mlsanity.com/paper/1812.10783/) (2018)

## Built on by (most cited first, 4 of 4)

- [ParamReL: Learning Parameter Space Representation via Progressively Encoding Bayesian Flow Networks](https://mlsanity.com/paper/2405.15268/) (2024)
- [Comparing the information content of probabilistic representation spaces](https://mlsanity.com/paper/2405.21042/) (2024)
- [Equivariant amortized inference of poses for cryo-EM](https://mlsanity.com/paper/2406.01630/) (2024)
- [Low-dimensional topology of deep neural networks](https://mlsanity.com/paper/2606.31856/) (2026)

## Abstract (arXiv)

Incorporating geometric inductive biases into models can aid interpretability and generalization, but encoding to a specific geometric structure can be challenging due to the imposed topological constraints. In this paper, we theoretically and empirically characterize obstructions to training encoders with geometric latent spaces. We show that local optima can arise due to singularities (e.g. self-intersection) or due to an incorrect degree or winding number. We then discuss how normalizing flows can potentially circumvent these obstructions by defining multimodal variational distributions. Inspired by this observation, we propose a new flow-based model that maps data points to multimodal distributions over geometric spaces and empirically evaluate our model on 2 domains. We observe improved stability during training and a higher chance of converging to a homeomorphic encoder.