# Capsule Networks Do Not Need to Model Everything

arXiv: https://arxiv.org/abs/2204.01298 · HTML: https://mlsanity.com/paper/2204.01298/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 18 of 18)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [Variational Dropout Sparsifies Deep Neural Networks](https://mlsanity.com/paper/1701.05369/) (2017)
- [Stacked Capsule Autoencoders](https://mlsanity.com/paper/1906.06818/) (2019)
- [DeepCaps: Going Deeper with Capsule Networks](https://mlsanity.com/paper/1904.09546/) (2019)
- [Efficient-CapsNet: Capsule Network with Self-Attention Routing](https://mlsanity.com/paper/2101.12491/) (2021)
- [Capsule Routing via Variational Bayes](https://mlsanity.com/paper/1905.11455/) (2019)
- [Learning Sparse Neural Networks via Sensitivity-Driven Regularization](https://mlsanity.com/paper/1810.11764/) (2018)
- [Improving the Robustness of Capsule Networks to Image Affine Transformations](https://mlsanity.com/paper/1911.07968/) (2019)
- [LOss-Based SensiTivity rEgulaRization: towards deep sparse neural networks](https://mlsanity.com/paper/2011.09905/) (2020)
- [Interpretable Graph Capsule Networks for Object Recognition](https://mlsanity.com/paper/2012.01674/) (2020)
- [Capsule Networks Need an Improved Routing Algorithm](https://mlsanity.com/paper/1907.13327/) (2019)
- [Neural Network Encapsulation](https://mlsanity.com/paper/1808.03749/) (2018)
- [Subspace Capsule Network](https://mlsanity.com/paper/2002.02924/) (2020)
- [Training Deep Capsule Networks with Residual Connections](https://mlsanity.com/paper/2104.07393/) (2021)

## Built on by (most cited first, 2 of 2)

- [OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning](https://mlsanity.com/paper/2403.13351/) (2024)
- [Hybrid Gromov-Wasserstein Embedding for Capsule Learning](https://mlsanity.com/paper/2209.00232/) (2022)

## Abstract (arXiv)

Capsule networks are biologically inspired neural networks that group neurons into vectors called capsules, each explicitly representing an object or one of its parts. The routing mechanism connects capsules in consecutive layers, forming a hierarchical structure between parts and objects, also known as a parse tree. Capsule networks often attempt to model all elements in an image, requiring large network sizes to handle complexities such as intricate backgrounds or irrelevant objects. However, this comprehensive modeling leads to increased parameter counts and computational inefficiencies. Our goal is to enable capsule networks to focus only on the object of interest, reducing the number of parse trees. We accomplish this with REM (Routing Entropy Minimization), a technique that minimizes the entropy of the parse tree-like structure. REM drives the model parameters distribution towards low entropy configurations through a pruning mechanism, significantly reducing the generation of intra-class parse trees. This empowers capsules to learn more stable and succinct representations with fewer parameters and negligible performance loss.