# A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exits

arXiv: https://arxiv.org/abs/2604.14789 · HTML: https://mlsanity.com/paper/2604.14789/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving.
- Builds on 26 of its 63 references that are in the map.
- First posted on arXiv 16 Apr 2026.

## Placed in

- Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/)

## Builds on (most cited first, 26 of 26)

- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [SkipNet: Learning Dynamic Routing in Convolutional Networks](https://mlsanity.com/paper/1711.09485/) (2017)
- [Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing](https://mlsanity.com/paper/1905.10083/) (2019)
- [QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving](https://mlsanity.com/paper/2405.04532/) (2024)
- [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](https://mlsanity.com/paper/2004.09576/) (2020)
- [Pruning vs Quantization: Which is Better?](https://mlsanity.com/paper/2307.02973/) (2023)
- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [SparseRT: Accelerating Unstructured Sparsity on GPUs for Deep Learning Inference](https://mlsanity.com/paper/2008.11849/) (2020)
- [Multi-Exit Vision Transformer for Dynamic Inference](https://mlsanity.com/paper/2106.15183/) (2021)
- [Balanced Sparsity for Efficient DNN Inference on GPU](https://mlsanity.com/paper/1811.00206/) (2018)
- [Efficient Execution of Quantized Deep Learning Models: A Compiler Approach](https://mlsanity.com/paper/2006.10226/) (2020)
- [Shallow-Deep Networks: Understanding and Mitigating Network Overthinking](https://mlsanity.com/paper/1810.07052/) (2018)
- [The Internet of Things, Fog and Cloud Continuum: Integration and Challenges](https://mlsanity.com/paper/1809.09972/) (2018)
- [QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference](https://mlsanity.com/paper/2402.10076/) (2024)
- [T-RECX: Tiny-Resource Efficient Convolutional neural networks with early-eXit](https://mlsanity.com/paper/2207.06613/) (2022)
- [PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices](https://mlsanity.com/paper/2310.19991/) (2023)
- [Edge AI: Evaluation of Model Compression Techniques for Convolutional Neural Networks](https://mlsanity.com/paper/2409.02134/) (2024)
- [EdgeAI: A Vision for Deep Learning in IoT Era](https://mlsanity.com/paper/1910.10356/) (2019)
- [QuantuneV2: Compiler-Based Local Metric-Driven Mixed Precision Quantization for Practical Embedded AI Applications](https://mlsanity.com/paper/2501.07161/) (2025)
- [Dynamic Shuffle: An Efficient Channel Mixture Method](https://mlsanity.com/paper/2310.02776/) (2023)
- [USDC: Unified Static and Dynamic Compression for Visual Transformer](https://mlsanity.com/paper/2310.11117/) (2023)

## Abstract (arXiv)

Deploying deep neural networks on edge devices requires balancing accuracy, latency, and resource constraints under realistic execution conditions. To fit models within these constraints, two broad strategies have emerged: static compression techniques such as pruning and quantization, which permanently reduce model size, and dynamic approaches such as early-exit mechanisms, which adapt computational cost at runtime. While both families are widely studied in isolation, they are rarely compared under identical conditions on physical hardware. This paper presents a unified deployment-oriented comparison of static compression and dynamic early-exit mechanisms, evaluated on real edge devices using ONNX based inference pipelines. Our results show that static and dynamic techniques offer fundamentally different trade-offs for edge deployment. While pruning and quantization deliver consistent memory footprint reduction, early-exit mechanisms enable input-adaptive computation savings that static methods cannot match. Their combination proves highly effective, simultaneously reducing inference latency and memory usage with minimal accuracy loss, expanding what is achievable at the edge.