# CapProNet: Deep Feature Learning via Orthogonal Projections onto Capsule Subspaces

arXiv: https://arxiv.org/abs/1805.07621 · HTML: https://mlsanity.com/paper/1805.07621/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2018.

## Built on by (most cited first, 9 of 9)

- [Effective and Efficient Vote Attack on Capsule Networks](https://mlsanity.com/paper/2102.10055/) (2021)
- [Associated Spatio-Temporal Capsule Network for Gait Recognition](https://mlsanity.com/paper/2101.02458/) (2021)
- [OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning](https://mlsanity.com/paper/2403.13351/) (2024)
- [Differentiable Projection for Constrained Deep Learning](https://mlsanity.com/paper/2111.10785/) (2021)
- [Grassmannian learning mutual subspace method for image set recognition](https://mlsanity.com/paper/2111.04352/) (2021)
- [DPR-CAE: Capsule Autoencoder with Dynamic Part Representation for Image Parsing](https://mlsanity.com/paper/2104.14735/) (2021)
- [Provable Clustering of a Union of Linear Manifolds Using Optimal Directions](https://mlsanity.com/paper/2201.02745/) (2022)
- [Effectiveness of the Recent Advances in Capsule Networks](https://mlsanity.com/paper/2210.05834/) (2022)
- [Capsule network with shortcut routing](https://mlsanity.com/paper/2307.10212/) (2023)

## Abstract (arXiv)

In this paper, we formalize the idea behind capsule nets of using a capsule vector rather than a neuron activation to predict the label of samples. To this end, we propose to learn a group of capsule subspaces onto which an input feature vector is projected. Then the lengths of resultant capsules are used to score the probability of belonging to different classes. We train such a Capsule Projection Network (CapProNet) by learning an orthogonal projection matrix for each capsule subspace, and show that each capsule subspace is updated until it contains input feature vectors corresponding to the associated class. We will also show that the capsule projection can be viewed as normalizing the multiple columns of the weight matrix simultaneously to form an orthogonal basis, which makes it more effective in incorporating novel components of input features to update capsule representations. In other words, the capsule projection can be viewed as a multi-dimensional weight normalization in capsule subspaces, where the conventional weight normalization is simply a special case of the capsule projection onto 1D lines. Only a small negligible computing overhead is incurred to train the network in low-dimensional capsule subspaces or through an alternative hyper-power iteration to estimate the normalization matrix. Experiment results on image datasets show the presented model can greatly improve the performance of the state-of-the-art ResNet backbones by 10-20% and that of the Densenet by 5-7% respectively at the same level of computing and memory expenses. The CapProNet establishes the competitive state-of-the-art performance for the family of capsule nets by significantly reducing test errors on the benchmark datasets.