# ProtoVAE: Prototypical Networks for Unsupervised Disentanglement

arXiv: https://arxiv.org/abs/2305.09092 · HTML: https://mlsanity.com/paper/2305.09092/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 16 May 2023.

## Builds on (most cited first, 24 of 24)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness](https://mlsanity.com/paper/1811.00007/) (2018)
- [On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset](https://mlsanity.com/paper/1906.03292/) (2019)
- [Deep Learning of Representations: Looking Forward](https://mlsanity.com/paper/1305.0445/) (2013)
- [Measuring Disentanglement: A Review of Metrics](https://mlsanity.com/paper/2012.09276/) (2020)
- [Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders](https://mlsanity.com/paper/1804.10469/) (2018)
- [Commutative Lie Group VAE for Disentanglement Learning](https://mlsanity.com/paper/2106.03375/) (2021)
- [Towards Building A Group-based Unsupervised Representation Disentanglement Framework](https://mlsanity.com/paper/2102.10303/) (2021)
- [Learning Disentangled Representations with Latent Variation Predictability](https://mlsanity.com/paper/2007.12885/) (2020)
- [Disentangling Factors of Variation by Mixing Them](https://mlsanity.com/paper/1711.07410/) (2017)
- [OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularization](https://mlsanity.com/paper/1905.10836/) (2019)
- [Dual Swap Disentangling](https://mlsanity.com/paper/1805.10583/) (2018)
- [Challenges in Disentangling Independent Factors of Variation](https://mlsanity.com/paper/1711.02245/) (2017)
- [A Cyclically-Trained Adversarial Network for Invariant Representation Learning](https://mlsanity.com/paper/1906.09313/) (2019)

## Abstract (arXiv)

Generative modeling and self-supervised learning have in recent years made great strides towards learning from data in a completely unsupervised way. There is still however an open area of investigation into guiding a neural network to encode the data into representations that are interpretable or explainable. The problem of unsupervised disentanglement is of particular importance as it proposes to discover the different latent factors of variation or semantic concepts from the data alone, without labeled examples, and encode them into structurally disjoint latent representations. Without additional constraints or inductive biases placed in the network, a generative model may learn the data distribution and encode the factors, but not necessarily in a disentangled way. Here, we introduce a novel deep generative VAE-based model, ProtoVAE, that leverages a deep metric learning Prototypical network trained using self-supervision to impose these constraints. The prototypical network constrains the mapping of the representation space to data space to ensure that controlled changes in the representation space are mapped to changes in the factors of variations in the data space. Our model is completely unsupervised and requires no a priori knowledge of the dataset, including the number of factors. We evaluate our proposed model on the benchmark dSprites, 3DShapes, and MPI3D disentanglement datasets, showing state of the art results against previous methods via qualitative traversals in the latent space, as well as quantitative disentanglement metrics. We further qualitatively demonstrate the effectiveness of our model on the real-world CelebA dataset.