# Towards Efficient Capsule Networks

arXiv: https://arxiv.org/abs/2208.09203 · HTML: https://mlsanity.com/paper/2208.09203/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 19 Aug 2022.

## Builds on (most cited first, 14 of 14)

- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [3D Point Capsule Networks](https://mlsanity.com/paper/1812.10775/) (2018)
- [EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning](https://mlsanity.com/paper/2007.02491/) (2020)
- [DeepCaps: Going Deeper with Capsule Networks](https://mlsanity.com/paper/1904.09546/) (2019)
- [Efficient-CapsNet: Capsule Network with Self-Attention Routing](https://mlsanity.com/paper/2101.12491/) (2021)
- [Learning Sparse Neural Networks via Sensitivity-Driven Regularization](https://mlsanity.com/paper/1810.11764/) (2018)
- [Brain Tumor Type Classification via Capsule Networks](https://mlsanity.com/paper/1802.10200/) (2018)
- [LOss-Based SensiTivity rEgulaRization: towards deep sparse neural networks](https://mlsanity.com/paper/2011.09905/) (2020)
- [SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks](https://mlsanity.com/paper/2102.03773/) (2021)
- [Capsule Networks Need an Improved Routing Algorithm](https://mlsanity.com/paper/1907.13327/) (2019)
- [Pruning artificial neural networks: a way to find well-generalizing, high-entropy sharp minima](https://mlsanity.com/paper/2004.14765/) (2020)
- [Q-CapsNets: A Specialized Framework for Quantizing Capsule Networks](https://mlsanity.com/paper/2004.07116/) (2020)
- [Training Deep Capsule Networks with Residual Connections](https://mlsanity.com/paper/2104.07393/) (2021)

## Built on by (most cited first, 3 of 3)

- [Can Unstructured Pruning Reduce the Depth in Deep Neural Networks?](https://mlsanity.com/paper/2308.06619/) (2023)
- [OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning](https://mlsanity.com/paper/2403.13351/) (2024)
- [When Does Pruning Benefit Vision Representations?](https://mlsanity.com/paper/2507.01722/) (2025)

## Abstract (arXiv)

From the moment Neural Networks dominated the scene for image processing, the computational complexity needed to solve the targeted tasks skyrocketed: against such an unsustainable trend, many strategies have been developed, ambitiously targeting performance's preservation. Promoting sparse topologies, for example, allows the deployment of deep neural networks models on embedded, resource-constrained devices. Recently, Capsule Networks were introduced to enhance explainability of a model, where each capsule is an explicit representation of an object or its parts. These models show promising results on toy datasets, but their low scalability prevents deployment on more complex tasks. In this work, we explore sparsity besides capsule representations to improve their computational efficiency by reducing the number of capsules. We show how pruning with Capsule Network achieves high generalization with less memory requirements, computational effort, and inference and training time.