# Improving the Robustness of Capsule Networks to Image Affine Transformations

arXiv: https://arxiv.org/abs/1911.07968 · HTML: https://mlsanity.com/paper/1911.07968/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2019.

## Built on by (most cited first, 21 of 21)

- [Efficient-CapsNet: Capsule Network with Self-Attention Routing](https://mlsanity.com/paper/2101.12491/) (2021)
- [Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models](https://mlsanity.com/paper/2306.02080/) (2023)
- [Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups](https://mlsanity.com/paper/2410.02698/) (2024)
- [Enabling equivariance for arbitrary Lie groups](https://mlsanity.com/paper/2111.08251/) (2021)
- [Which Model Generated This Image? A Model-Agnostic Approach for Origin Attribution](https://mlsanity.com/paper/2404.02697/) (2024)
- [Effective and Efficient Vote Attack on Capsule Networks](https://mlsanity.com/paper/2102.10055/) (2021)
- [Reliable Evaluation of Adversarial Transferability](https://mlsanity.com/paper/2306.08565/) (2023)
- [Capsule Network is Not More Robust than Convolutional Network](https://mlsanity.com/paper/2103.15459/) (2021)
- [Lie Group Decompositions for Equivariant Neural Networks](https://mlsanity.com/paper/2310.11366/) (2023)
- [3D-UCaps: 3D Capsules Unet for Volumetric Image Segmentation](https://mlsanity.com/paper/2203.08965/) (2022)
- [Towards Robust Prompts on Vision-Language Models](https://mlsanity.com/paper/2304.08479/) (2023)
- [Why Capsule Neural Networks Do Not Scale: Challenging the Dynamic Parse-Tree Assumption](https://mlsanity.com/paper/2301.01583/) (2023)
- [XnODR and XnIDR: Two Accurate and Fast Fully Connected Layers For Convolutional Neural Networks](https://mlsanity.com/paper/2111.10854/) (2021)
- [3DConvCaps: 3DUnet with Convolutional Capsule Encoder for Medical Image Segmentation](https://mlsanity.com/paper/2205.09299/) (2022)
- [Surface Defect Detection and Evaluation for Marine Vessels using Multi-Stage Deep Learning](https://mlsanity.com/paper/2203.09580/) (2022)
- [Capsule Networks Do Not Need to Model Everything](https://mlsanity.com/paper/2204.01298/) (2022)
- [RobCaps: Evaluating the Robustness of Capsule Networks against Affine Transformations and Adversarial Attacks](https://mlsanity.com/paper/2304.03973/) (2023)
- [CapsNet for Medical Image Segmentation](https://mlsanity.com/paper/2203.08948/) (2022)
- [Single Image Super-Resolution Based on Capsule Neural Networks](https://mlsanity.com/paper/2210.03743/) (2022)
- [Explainability and Robustness of Deep Visual Classification Models](https://mlsanity.com/paper/2301.01343/) (2023)
- [QSV: Quat-Sphere-Vision for Coupled Quaternion Attention on Spherical Lattices](https://mlsanity.com/paper/2609.30592/) (2026)

## Abstract (arXiv)

Convolutional neural networks (CNNs) achieve translational invariance by using pooling operations. However, the operations do not preserve the spatial relationships in the learned representations. Hence, CNNs cannot extrapolate to various geometric transformations of inputs. Recently, Capsule Networks (CapsNets) have been proposed to tackle this problem. In CapsNets, each entity is represented by a vector and routed to high-level entity representations by a dynamic routing algorithm. CapsNets have been shown to be more robust than CNNs to affine transformations of inputs. However, there is still a huge gap between their performance on transformed inputs compared to untransformed versions. In this work, we first revisit the routing procedure by (un)rolling its forward and backward passes. Our investigation reveals that the routing procedure contributes neither to the generalization ability nor to the affine robustness of the CapsNets. Furthermore, we explore the limitations of capsule transformations and propose affine CapsNets (Aff-CapsNets), which are more robust to affine transformations. On our benchmark task, where models are trained on the MNIST dataset and tested on the AffNIST dataset, our Aff-CapsNets improve the benchmark performance by a large margin (from 79% to 93.21%), without using any routing mechanism.