# Shifting Capsule Networks from the Cloud to the Deep Edge

arXiv: https://arxiv.org/abs/2110.02911 · HTML: https://mlsanity.com/paper/2110.02911/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 6 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation](https://mlsanity.com/paper/2004.09602/) (2020)
- [CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs](https://mlsanity.com/paper/1801.06601/) (2018)
- [Stacked Capsule Autoencoders](https://mlsanity.com/paper/1906.06818/) (2019)
- [Learning to Perform Physics Experiments via Deep Reinforcement Learning](https://mlsanity.com/paper/1611.01843/) (2016)
- [PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processors](https://mlsanity.com/paper/1908.11263/) (2019)
- [Quantization and Deployment of Deep Neural Networks on Microcontrollers](https://mlsanity.com/paper/2105.13331/) (2021)
- [A Berkeley View of Systems Challenges for AI](https://mlsanity.com/paper/1712.05855/) (2017)
- [Q-CapsNets: A Specialized Framework for Quantizing Capsule Networks](https://mlsanity.com/paper/2004.07116/) (2020)
- [Novel Deep Learning Model for Traffic Sign Detection Using Capsule Networks](https://mlsanity.com/paper/1805.04424/) (2018)
- [Towards Linear Time Neural Machine Translation with Capsule Networks](https://mlsanity.com/paper/1811.00287/) (2018)
- [DESCNet: Developing Efficient Scratchpad Memories for Capsule Network Hardware](https://mlsanity.com/paper/2010.05754/) (2020)
- [Galaxy morphology prediction using capsule networks](https://mlsanity.com/paper/1809.08377/) (2018)
- [Enabling Highly Efficient Capsule Networks Processing Through A PIM-Based Architecture Design](https://mlsanity.com/paper/1911.03451/) (2019)

## Built on by (most cited first, 2 of 2)

- [Decentor-V: Lightweight ML Training on Low-Power RISC-V Edge Devices](https://mlsanity.com/paper/2509.18118/) (2025)
- [David and Goliath: An Empirical Evaluation of Attacks and Defenses for QNNs at the Deep Edge](https://mlsanity.com/paper/2404.05688/) (2024)

## Abstract (arXiv)

Capsule networks (CapsNets) are an emerging trend in image processing. In contrast to a convolutional neural network, CapsNets are not vulnerable to object deformation, as the relative spatial information of the objects is preserved across the network. However, their complexity is mainly related to the capsule structure and the dynamic routing mechanism, which makes it almost unreasonable to deploy a CapsNet, in its original form, in a resource-constrained device powered by a small microcontroller (MCU). In an era where intelligence is rapidly shifting from the cloud to the edge, this high complexity imposes serious challenges to the adoption of CapsNets at the very edge. To tackle this issue, we present an API for the execution of quantized CapsNets in Arm Cortex-M and RISC-V MCUs. Our software kernels extend the Arm CMSIS-NN and RISC-V PULP-NN to support capsule operations with 8-bit integers as operands. Along with it, we propose a framework to perform post-training quantization of a CapsNet. Results show a reduction in memory footprint of almost 75%, with accuracy loss ranging from 0.07% to 0.18%. In terms of throughput, our Arm Cortex-M API enables the execution of primary capsule and capsule layers with medium-sized kernels in just 119.94 and 90.60 milliseconds (ms), respectively (STM32H755ZIT6U, Cortex-M7 @ 480 MHz). For the GAP-8 SoC (RISC-V RV32IMCXpulp @ 170 MHz), the latency drops to 7.02 and 38.03 ms, respectively.