# Training Deep Capsule Networks with Residual Connections

arXiv: https://arxiv.org/abs/2104.07393 · HTML: https://mlsanity.com/paper/2104.07393/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [A Survey on Neural Architecture Search](https://mlsanity.com/paper/1905.01392/) (2019)
- [DeepCaps: Going Deeper with Capsule Networks](https://mlsanity.com/paper/1904.09546/) (2019)
- [Capsule Routing via Variational Bayes](https://mlsanity.com/paper/1905.11455/) (2019)
- [Capsules with Inverted Dot-Product Attention Routing](https://mlsanity.com/paper/2002.04764/) (2020)
- [Capsule Network Performance on Complex Data](https://mlsanity.com/paper/1712.03480/) (2017)
- [Increasing the adversarial robustness and explainability of capsule networks with γ-capsules](https://mlsanity.com/paper/1812.09707/) (2018)
- [Conflicting Bundles: Adapting Architectures Towards the Improved Training of Deep Neural Networks](https://mlsanity.com/paper/2011.02956/) (2020)
- [Auto-tuning of Deep Neural Networks by Conflicting Layer Removal](https://mlsanity.com/paper/2103.04331/) (2021)

## Built on by (most cited first, 4 of 4)

- [Towards Efficient Capsule Networks](https://mlsanity.com/paper/2208.09203/) (2022)
- [Vanishing Activations: A Symptom of Deep Capsule Networks](https://mlsanity.com/paper/2305.11178/) (2023)
- [Capsule Networks Do Not Need to Model Everything](https://mlsanity.com/paper/2204.01298/) (2022)
- [Momentum Capsule Networks](https://mlsanity.com/paper/2201.11091/) (2022)

## Abstract (arXiv)

Capsule networks are a type of neural network that have recently gained increased popularity. They consist of groups of neurons, called capsules, which encode properties of objects or object parts. The connections between capsules encrypt part-whole relationships between objects through routing algorithms which route the output of capsules from lower level layers to upper level layers. Capsule networks can reach state-of-the-art results on many challenging computer vision tasks, such as MNIST, Fashion-MNIST, and Small-NORB. However, most capsule network implementations use two to three capsule layers, which limits their applicability as expressivity grows exponentially with depth. One approach to overcome such limitations would be to train deeper network architectures, as it has been done for convolutional neural networks with much increased success. In this paper, we propose a methodology to train deeper capsule networks using residual connections, which is evaluated on four datasets and three different routing algorithms. Our experimental results show that in fact, performance increases when training deeper capsule networks. The source code is available on https://github.com/moejoe95/res-capsnet.