# Topographic VAEs learn Equivariant Capsules

arXiv: https://arxiv.org/abs/2109.01394 · HTML: https://mlsanity.com/paper/2109.01394/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 54 references that are in the map.
- First posted on arXiv 3 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [Equivariance Through Parameter-Sharing](https://mlsanity.com/paper/1702.08389/) (2017)
- [Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding](https://mlsanity.com/paper/2007.10930/) (2020)
- [Learning Invariances in Neural Networks](https://mlsanity.com/paper/2010.11882/) (2020)
- [Stochastic Backpropagation and Approximate Inference in Deep Generative Models](https://mlsanity.com/paper/1401.4082/) (2014)
- [Group Equivariant Capsule Networks](https://mlsanity.com/paper/1806.05086/) (2018)
- [Transformation Properties of Learned Visual Representations](https://mlsanity.com/paper/1412.7659/) (2014)
- [Addressing the Topological Defects of Disentanglement via Distributed Operators](https://mlsanity.com/paper/2102.05623/) (2021)
- [Learning to Convolve: A Generalized Weight-Tying Approach](https://mlsanity.com/paper/1905.04663/) (2019)
- [Disentangling Space and Time in Video with Hierarchical Variational Auto-encoders](https://mlsanity.com/paper/1612.04440/) (2016)
- [Variational Autoencoder with Embedded Student-t Mixture Model for Authorship Attribution](https://mlsanity.com/paper/2005.13930/) (2020)

## Abstract (arXiv)

In this work we seek to bridge the concepts of topographic organization and equivariance in neural networks. To accomplish this, we introduce the Topographic VAE: a novel method for efficiently training deep generative models with topographically organized latent variables. We show that such a model indeed learns to organize its activations according to salient characteristics such as digit class, width, and style on MNIST. Furthermore, through topographic organization over time (i.e. temporal coherence), we demonstrate how predefined latent space transformation operators can be encouraged for observed transformed input sequences -- a primitive form of unsupervised learned equivariance. We demonstrate that this model successfully learns sets of approximately equivariant features (i.e. "capsules") directly from sequences and achieves higher likelihood on correspondingly transforming test sequences. Equivariance is verified quantitatively by measuring the approximate commutativity of the inference network and the sequence transformations. Finally, we demonstrate approximate equivariance to complex transformations, expanding upon the capabilities of existing group equivariant neural networks.