# Efficient-CapsNet: Capsule Network with Self-Attention Routing

arXiv: https://arxiv.org/abs/2101.12491 · HTML: https://mlsanity.com/paper/2101.12491/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 29 Jan 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [Capsule Routing via Variational Bayes](https://mlsanity.com/paper/1905.11455/) (2019)
- [Group Equivariant Capsule Networks](https://mlsanity.com/paper/1806.05086/) (2018)
- [Capsules for Object Segmentation](https://mlsanity.com/paper/1804.04241/) (2018)
- [VideoCapsuleNet: A Simplified Network for Action Detection](https://mlsanity.com/paper/1805.08162/) (2018)
- [Capsules with Inverted Dot-Product Attention Routing](https://mlsanity.com/paper/2002.04764/) (2020)
- [Capsule Network Performance on Complex Data](https://mlsanity.com/paper/1712.03480/) (2017)
- [Multi-image Super Resolution of Remotely Sensed Images using Residual Feature Attention Deep Neural Networks](https://mlsanity.com/paper/2007.03107/) (2020)
- [Improving the Robustness of Capsule Networks to Image Affine Transformations](https://mlsanity.com/paper/1911.07968/) (2019)
- [Attention routing between capsules](https://mlsanity.com/paper/1907.01750/) (2019)
- [RMDL: Random Multimodel Deep Learning for Classification](https://mlsanity.com/paper/1805.01890/) (2018)
- [Capsule Networks Need an Improved Routing Algorithm](https://mlsanity.com/paper/1907.13327/) (2019)
- [Real-Time Apple Detection System Using Embedded Systems With Hardware Accelerators: An Edge AI Application](https://mlsanity.com/paper/2004.13410/) (2020)
- [Attention-Based Capsule Networks with Dynamic Routing for Relation Extraction](https://mlsanity.com/paper/1812.11321/) (2018)
- [CapsuleGAN: Generative Adversarial Capsule Network](https://mlsanity.com/paper/1802.06167/) (2018)
- [Learning Compositional Structures for Deep Learning: Why Routing-by-agreement is Necessary](https://mlsanity.com/paper/2010.01488/) (2020)

## Built on by (most cited first, 30 of 41)

- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes](https://mlsanity.com/paper/2110.08059/) (2021)
- [Enabling equivariance for arbitrary Lie groups](https://mlsanity.com/paper/2111.08251/) (2021)
- [Capsule Network is Not More Robust than Convolutional Network](https://mlsanity.com/paper/2103.15459/) (2021)
- [SwinCheX: Multi-label classification on chest X-ray images with transformers](https://mlsanity.com/paper/2206.04246/) (2022)
- [Interpretable part-whole hierarchies and conceptual-semantic relationships in neural networks](https://mlsanity.com/paper/2203.03282/) (2022)
- [Weakly Supervised Grounding for VQA in Vision-Language Transformers](https://mlsanity.com/paper/2207.02334/) (2022)
- [Tighter Abstract Queries in Neural Network Verification](https://mlsanity.com/paper/2210.12871/) (2022)
- [An Overview of Recent Work in Media Forensics: Methods and Threats](https://mlsanity.com/paper/2204.12067/) (2022)
- [Spiking CapsNet: A Spiking Neural Network With A Biologically Plausible Routing Rule Between Capsules](https://mlsanity.com/paper/2111.07785/) (2021)
- [Sampling Agnostic Feature Representation for Long-Term Person Re-identification](https://mlsanity.com/paper/2209.09574/) (2022)
- [Why Capsule Neural Networks Do Not Scale: Challenging the Dynamic Parse-Tree Assumption](https://mlsanity.com/paper/2301.01583/) (2023)
- [SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features](https://mlsanity.com/paper/2309.16987/) (2023)
- [XnODR and XnIDR: Two Accurate and Fast Fully Connected Layers For Convolutional Neural Networks](https://mlsanity.com/paper/2111.10854/) (2021)
- [Routing with Self-Attention for Multimodal Capsule Networks](https://mlsanity.com/paper/2112.00775/) (2021)
- [Convolutional Neural Networks as 2-D systems](https://mlsanity.com/paper/2303.03042/) (2023)
- [OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning](https://mlsanity.com/paper/2403.13351/) (2024)
- [Gaussian Mixture Convolution Networks](https://mlsanity.com/paper/2202.09153/) (2022)
- [Robust Self-Supervised Learning with Lie Groups](https://mlsanity.com/paper/2210.13356/) (2022)
- [ProtoCaps: A Fast and Non-Iterative Capsule Network Routing Method](https://mlsanity.com/paper/2307.09944/) (2023)
- [Towards Efficient Capsule Networks](https://mlsanity.com/paper/2208.09203/) (2022)
- [Modularity in Deep Learning: A Survey](https://mlsanity.com/paper/2310.01154/) (2023)
- [CvS: Classification via Segmentation For Small Datasets](https://mlsanity.com/paper/2111.00042/) (2021)
- [Capsule Networks Do Not Need to Model Everything](https://mlsanity.com/paper/2204.01298/) (2022)
- [Capsule Network Projectors are Equivariant and Invariant Learners](https://mlsanity.com/paper/2405.14386/) (2024)
- [Retinal Vessel Segmentation with Deep Graph and Capsule Reasoning](https://mlsanity.com/paper/2409.11508/) (2024)
- [Hybrid Gromov-Wasserstein Embedding for Capsule Learning](https://mlsanity.com/paper/2209.00232/) (2022)
- [Parametric Matrix Models](https://mlsanity.com/paper/2401.11694/) (2024)
- [SVarM: Linear Support Varifold Machines for Classification and Regression on Geometric Data](https://mlsanity.com/paper/2506.01189/) (2025)
- [SUTD-PRCM Dataset and Neural Architecture Search Approach for Complex Metasurface Design](https://mlsanity.com/paper/2203.00002/) (2022)

## Abstract (arXiv)

Deep convolutional neural networks, assisted by architectural design strategies, make extensive use of data augmentation techniques and layers with a high number of feature maps to embed object transformations. That is highly inefficient and for large datasets implies a massive redundancy of features detectors. Even though capsules networks are still in their infancy, they constitute a promising solution to extend current convolutional networks and endow artificial visual perception with a process to encode more efficiently all feature affine transformations. Indeed, a properly working capsule network should theoretically achieve higher results with a considerably lower number of parameters count due to intrinsic capability to generalize to novel viewpoints. Nevertheless, little attention has been given to this relevant aspect. In this paper, we investigate the efficiency of capsule networks and, pushing their capacity to the limits with an extreme architecture with barely 160K parameters, we prove that the proposed architecture is still able to achieve state-of-the-art results on three different datasets with only 2% of the original CapsNet parameters. Moreover, we replace dynamic routing with a novel non-iterative, highly parallelizable routing algorithm that can easily cope with a reduced number of capsules. Extensive experimentation with other capsule implementations has proved the effectiveness of our methodology and the capability of capsule networks to efficiently embed visual representations more prone to generalization.