# Attention routing between capsules

arXiv: https://arxiv.org/abs/1907.01750 · HTML: https://mlsanity.com/paper/1907.01750/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2019.

## Built on by (most cited first, 17 of 17)

- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Efficient-CapsNet: Capsule Network with Self-Attention Routing](https://mlsanity.com/paper/2101.12491/) (2021)
- [Concept Graph Neural Networks for Surgical Video Understanding](https://mlsanity.com/paper/2202.13402/) (2022)
- [Spiking CapsNet: A Spiking Neural Network With A Biologically Plausible Routing Rule Between Capsules](https://mlsanity.com/paper/2111.07785/) (2021)
- [Graph Routing between Capsules](https://mlsanity.com/paper/2106.11531/) (2021)
- [OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning](https://mlsanity.com/paper/2403.13351/) (2024)
- [Retinal Vessel Segmentation with Deep Graph and Capsule Reasoning](https://mlsanity.com/paper/2409.11508/) (2024)
- [Hybrid Gromov-Wasserstein Embedding for Capsule Learning](https://mlsanity.com/paper/2209.00232/) (2022)
- [RobCaps: Evaluating the Robustness of Capsule Networks against Affine Transformations and Adversarial Attacks](https://mlsanity.com/paper/2304.03973/) (2023)
- [ParseCaps: An Interpretable Parsing Capsule Network for Medical Image Diagnosis](https://mlsanity.com/paper/2411.01564/) (2024)
- [How to Accelerate Capsule Convolutions in Capsule Networks](https://mlsanity.com/paper/2104.02621/) (2021)
- [Capsule networks with non-iterative cluster routing](https://mlsanity.com/paper/2109.09213/) (2021)
- [ME-CapsNet: A Multi-Enhanced Capsule Networks with Routing Mechanism](https://mlsanity.com/paper/2203.15547/) (2022)
- [Iterative collaborative routing among equivariant capsules for transformation-robust capsule networks](https://mlsanity.com/paper/2210.11095/) (2022)
- [A Tensor-based Convolutional Neural Network for Small Dataset Classification](https://mlsanity.com/paper/2303.17061/) (2023)
- [Capsule network with shortcut routing](https://mlsanity.com/paper/2307.10212/) (2023)
- [MSPCaps: A Multi-Scale Patchify Capsule Network with Cross-Agreement Routing for Visual Recognition](https://mlsanity.com/paper/2508.16922/) (2025)

## Abstract (arXiv)

In this paper, we propose a new capsule network architecture called Attention Routing CapsuleNet (AR CapsNet). We replace the dynamic routing and squash activation function of the capsule network with dynamic routing (CapsuleNet) with the attention routing and capsule activation. The attention routing is a routing between capsules through an attention module. The attention routing is a fast forward-pass while keeping spatial information. On the other hand, the intuitive interpretation of the dynamic routing is finding a centroid of the prediction capsules. Thus, the squash activation function and its variant focus on preserving a vector orientation. However, the capsule activation focuses on performing a capsule-scale activation function. We evaluate our proposed model on the MNIST, affNIST, and CIFAR-10 classification tasks. The proposed model achieves higher accuracy with fewer parameters (x0.65 in the MNIST, x0.82 in the CIFAR-10) and less training time than CapsuleNet (x0.19 in the MNIST, x0.35 in the CIFAR-10). These results validate that designing a capsule-scale operation is a key factor to implement the capsule concept. Also, our experiment shows that our proposed model is transformation equivariant as CapsuleNet. As we perturb each element of the output capsule, the decoder attached to the output capsules shows global variations. Further experiments show that the difference in the capsule features caused by applying affine transformations on an input image is significantly aligned in one direction.