# Sparse Unsupervised Capsules Generalize Better

arXiv: https://arxiv.org/abs/1804.06094 · HTML: https://mlsanity.com/paper/1804.06094/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2018.

## Built on by (most cited first, 8 of 8)

- [Enabling equivariance for arbitrary Lie groups](https://mlsanity.com/paper/2111.08251/) (2021)
- [Effective and Efficient Vote Attack on Capsule Networks](https://mlsanity.com/paper/2102.10055/) (2021)
- [Capsule Network is Not More Robust than Convolutional Network](https://mlsanity.com/paper/2103.15459/) (2021)
- [Why Capsule Neural Networks Do Not Scale: Challenging the Dynamic Parse-Tree Assumption](https://mlsanity.com/paper/2301.01583/) (2023)
- [A Novel Capsule Neural Network Based Model for Drowsiness Detection Using Electroencephalography Signals](https://mlsanity.com/paper/2204.01666/) (2022)
- [Effectiveness of the Recent Advances in Capsule Networks](https://mlsanity.com/paper/2210.05834/) (2022)
- [CapsFlow: Optical Flow Estimation with Capsule Networks](https://mlsanity.com/paper/2304.00306/) (2023)
- [Bhargava Cube--Inspired Quadratic Regularization for Structured Neural Embeddings](https://mlsanity.com/paper/2512.11392/) (2025)

## Abstract (arXiv)

We show that unsupervised training of latent capsule layers using only the reconstruction loss, without masking to select the correct output class, causes a loss of equivariances and other desirable capsule qualities. This implies that supervised capsules networks can't be very deep. Unsupervised sparsening of latent capsule layer activity both restores these qualities and appears to generalize better than supervised masking, while potentially enabling deeper capsules networks. We train a sparse, unsupervised capsules network of similar geometry to Sabour et al (2017) on MNIST, and then test classification accuracy on affNIST using an SVM layer. Accuracy is improved from benchmark 79% to 90%.