# Deformable Capsules for Object Detection

arXiv: https://arxiv.org/abs/2104.05031 · HTML: https://mlsanity.com/paper/2104.05031/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 47 references that are in the map.
- First posted on arXiv 11 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Scale-Aware Trident Networks for Object Detection](https://mlsanity.com/paper/1901.01892/) (2019)
- [Bottom-up Object Detection by Grouping Extreme and Center Points](https://mlsanity.com/paper/1901.08043/) (2019)
- [Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects](https://mlsanity.com/paper/1811.11553/) (2018)
- [Single-Shot Refinement Neural Network for Object Detection](https://mlsanity.com/paper/1711.06897/) (2017)
- [3D Point Capsule Networks](https://mlsanity.com/paper/1812.10775/) (2018)
- [SNIPER: Efficient Multi-Scale Training](https://mlsanity.com/paper/1805.09300/) (2018)
- [Stacked Capsule Autoencoders](https://mlsanity.com/paper/1906.06818/) (2019)
- [Capsules for Object Segmentation](https://mlsanity.com/paper/1804.04241/) (2018)
- [VideoCapsuleNet: A Simplified Network for Action Detection](https://mlsanity.com/paper/1805.08162/) (2018)
- [CapsuleVOS: Semi-Supervised Video Object Segmentation Using Capsule Routing](https://mlsanity.com/paper/1910.00132/) (2019)
- [Capsules with Inverted Dot-Product Attention Routing](https://mlsanity.com/paper/2002.04764/) (2020)

## Built on by (most cited first, 3 of 3)

- [Mamba Capsule Routing Towards Part-Whole Relational Camouflaged Object Detection](https://mlsanity.com/paper/2410.03987/) (2024)
- [Retinal Vessel Segmentation with Deep Graph and Capsule Reasoning](https://mlsanity.com/paper/2409.11508/) (2024)
- [Hybrid Gromov-Wasserstein Embedding for Capsule Learning](https://mlsanity.com/paper/2209.00232/) (2022)

## Abstract (arXiv)

Capsule networks promise significant benefits over convolutional networks by storing stronger internal representations, and routing information based on the agreement between intermediate representations' projections. Despite this, their success has been limited to small-scale classification datasets due to their computationally expensive nature. Though memory efficient, convolutional capsules impose geometric constraints that fundamentally limit the ability of capsules to model the pose/deformation of objects. Further, they do not address the bigger memory concern of class-capsules scaling up to bigger tasks such as detection or large-scale classification. In this study, we introduce a new family of capsule networks, deformable capsules (textitDeformCaps), to address a very important problem in computer vision: object detection. We propose two new algorithms associated with our textitDeformCaps: a novel capsule structure (textitSplitCaps), and a novel dynamic routing algorithm (textitSE-Routing), which balance computational efficiency with the need for modeling a large number of objects and classes, which have never been achieved with capsule networks before. We demonstrate that the proposed methods efficiently scale up to create the first-ever capsule network for object detection in the literature. Our proposed architecture is a one-stage detection framework and it obtains results on MS COCO which are on par with state-of-the-art one-stage CNN-based methods, while producing fewer false positive detection, generalizing to unusual poses/viewpoints of objects.