# β-CapsNet: Learning Disentangled Representation for CapsNet by Information Bottleneck

arXiv: https://arxiv.org/abs/2209.05239 · HTML: https://mlsanity.com/paper/2209.05239/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 12 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [Information Dropout: Learning Optimal Representations Through Noisy Computation](https://mlsanity.com/paper/1611.01353/) (2016)
- [Deep Convolutional Inverse Graphics Network](https://mlsanity.com/paper/1503.03167/) (2015)
- [Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow](https://mlsanity.com/paper/1810.00821/) (2018)
- [Learning Disentangled Joint Continuous and Discrete Representations](https://mlsanity.com/paper/1804.00104/) (2018)
- [Stacked Capsule Autoencoders](https://mlsanity.com/paper/1906.06818/) (2019)
- [DeepCaps: Going Deeper with Capsule Networks](https://mlsanity.com/paper/1904.09546/) (2019)
- [Capsule Routing via Variational Bayes](https://mlsanity.com/paper/1905.11455/) (2019)
- [Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis](https://mlsanity.com/paper/1601.00706/) (2016)
- [A Survey of Inductive Biases for Factorial Representation-Learning](https://mlsanity.com/paper/1612.05299/) (2016)
- [Relevant sparse codes with variational information bottleneck](https://mlsanity.com/paper/1605.07332/) (2016)
- [Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement](https://mlsanity.com/paper/1902.05770/) (2019)
- [Neural Network Encapsulation](https://mlsanity.com/paper/1808.03749/) (2018)

## Built on by (most cited first, 1 of 1)

- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)

## Abstract (arXiv)

We present a framework for learning disentangled representation of CapsNet by information bottleneck constraint that distills information into a compact form and motivates to learn an interpretable factorized capsule. In our β-CapsNet framework, hyperparameter β is utilized to trade-off disentanglement and other tasks, variational inference is utilized to convert the information bottleneck term into a KL divergence that is approximated as a constraint on the mean of the capsule. For supervised learning, class independent mask vector is used for understanding the types of variations synthetically irrespective of the image class, we carry out extensive quantitative and qualitative experiments by tuning the parameter β to figure out the relationship between disentanglement, reconstruction and classfication performance. Furthermore, the unsupervised β-CapsNet and the corresponding dynamic routing algorithm is proposed for learning disentangled capsule in an unsupervised manner, extensive empirical evaluations suggest that our β-CapsNet achieves state-of-the-art disentanglement performance compared to CapsNet and various baselines on several complex datasets both in supervision and unsupervised scenes.