# Adaptive AI: Energy Efficient Multi-exit TinyML on Intelligent Vision Systems at the Edge

arXiv: https://arxiv.org/abs/2609.11939 · HTML: https://mlsanity.com/paper/2609.11939/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 7 Jul 2026.

## Builds on (most cited first, 21 of 21)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [MaxViT: Multi-Axis Vision Transformer](https://mlsanity.com/paper/2204.01697/) (2022)
- [SkipNet: Learning Dynamic Routing in Convolutional Networks](https://mlsanity.com/paper/1711.09485/) (2017)
- [Dynamic Early Exit in Reasoning Models](https://mlsanity.com/paper/2504.15895/) (2025)
- [Why should we add early exits to neural networks?](https://mlsanity.com/paper/2004.12814/) (2020)
- [Tiny Machine Learning: Progress and Futures](https://mlsanity.com/paper/2403.19076/) (2024)
- [Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference](https://mlsanity.com/paper/2206.04685/) (2022)
- [E2CM: Early Exit via Class Means for Efficient Supervised and Unsupervised Learning](https://mlsanity.com/paper/2103.01148/) (2021)
- [A Sim-to-Real Deep Learning-based Framework for Autonomous Nano-drone Racing](https://mlsanity.com/paper/2312.08991/) (2023)
- [Distilling Tiny and Ultra-fast Deep Neural Networks for Autonomous Navigation on Nano-UAVs](https://mlsanity.com/paper/2407.12675/) (2024)
- [Sim-to-Real Vision-depth Fusion CNNs for Robust Pose Estimation Aboard Autonomous Nano-quadcopter](https://mlsanity.com/paper/2308.01833/) (2023)
- [Multi-resolution Rescored ByteTrack for Video Object Detection on Ultra-low-power Embedded Systems](https://mlsanity.com/paper/2404.11488/) (2024)
- [QUTE: Quantifying Uncertainty in TinyML with Early-exit-assisted ensembles for model-monitoring](https://mlsanity.com/paper/2404.12599/) (2024)
- [DyCE: Dynamically Configurable Exiting for Deep Learning Compression and Real-time Scaling](https://mlsanity.com/paper/2403.01695/) (2024)
- [A Deep Learning-based Pest Insect Monitoring System for Ultra-low Power Pocket-sized Drones](https://mlsanity.com/paper/2407.00815/) (2024)
- [High-throughput Visual Nano-drone to Nano-drone Relative Localization using Onboard Fully Convolutional Networks](https://mlsanity.com/paper/2402.13756/) (2024)
- [Deep Feature Surgery: Towards Accurate and Efficient Multi-Exit Networks](https://mlsanity.com/paper/2407.13986/) (2024)

## Abstract (arXiv)

Traditional TinyML systems for edge devices achieve high accuracy by relying on fixed-depth models that require a constant number of multiply-accumulate (MAC) operations regardless of the input complexity. This approach wastes critical resources in battery-powered Internet-of-Things (IoT) devices and limits the real-time performance of edge cyber-physical systems. Multi-exit execution schemes mitigate these issues and are widely used on high-end devices such as GPUs, but are rarely exploited on edge IoT devices because they require substantial rethinking given their strict memory and computational constraints. We address these aspects by designing and deploying, on an ultra-low-power GWT GAP9 System-on-Chip (SoC), a novel multi-exit computational scheme, demonstrating it on a MobileNetV2 convolutional neural network (CNN) for the ImageNet-100 classification task. Our approach introduces multiple exits at different CNN depths, each with a confidence-based gating mechanism that dynamically and autonomously decides whether to continue or stop inference. Comparing our multi-exit strategy to the standard MobileNetV2 on a GAP9 SoC, we show a 41% reduction in the average computational cost (from 313 MMAC to 185 MMAC), a 29% lower inference time (from 49 to 35 ms), and an energy saving of 24% (from 2.1 to 1.6 mJ per frame). All these improvements come with a ~1% loss in accuracy compared to the full-depth MobileNetV2, which achieves 80.5%. Finally, comparing our adaptable multi-exit scheme with a third-party state-of-the-art adaptive CNN, also deployed on the GAP9, we achieve more than 2x its computational efficiency, increasing it from 8.1 to 17.2 MAC/cycle.