mlsanityData to 14 Oct 2026

A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exits

arXiv 2604.14789 ↗First posted 16 Apr 2026cs.AIcited by 0

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Deploying deep neural networks on edge devices requires balancing accuracy, latency, and resource constraints under realistic execution conditions. To fit models within these constraints, two broad strategies have emerged: static compression techniques such as pruning and quantization, which permanently reduce model size, and dynamic approaches such as early-exit mechanisms, which adapt computational cost at runtime. While both families are widely studied in isolation, they are rarely compared under identical conditions on physical hardware. This paper presents a unified deployment-oriented comparison of static compression and dynamic early-exit mechanisms, evaluated on real edge devices using ONNX based inference pipelines. Our results show that static and dynamic techniques offer fundamentally different trade-offs for edge deployment. While pruning and quantization deliver consistent memory footprint reduction, early-exit mechanisms enable input-adaptive computation savings that static methods cannot match. Their combination proves highly effective, simultaneously reducing inference latency and memory usage with minimal accuracy loss, expanding what is achievable at the edge.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 26 of its 63 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

26 of 63 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022T-RECX: Tiny-Resource Efficient Convolutional neural networks with early-eXit14 Jul
  2. 2021Multi-Exit Vision Transformer for Dynamic Inference29 Jun

Other references · 24

  1. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  2. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  3. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  4. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  5. 2021A White Paper on Neural Network Quantizationcited by 472
  6. 2021Dynamic Neural Networks: A Surveycited by 408
  7. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 332
  8. 2019Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computingcited by 228
  9. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
  10. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 189
  11. 2023Pruning vs Quantization: Which is Better?cited by 46
  12. 2025Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Modelscited by 46
12 more
  1. 2020SparseRT: Accelerating Unstructured Sparsity on GPUs for Deep Learning Inferencecited by 30
  2. 2018Balanced Sparsity for Efficient DNN Inference on GPUcited by 27
  3. 2020Efficient Execution of Quantized Deep Learning Models: A Compiler Approachcited by 13
  4. 2018Shallow-Deep Networks: Understanding and Mitigating Network Overthinkingcited by 7
  5. 2018The Internet of Things, Fog and Cloud Continuum: Integration and Challengescited by 6
  6. 2024QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inferencecited by 6
  7. 2023PolyThrottle: Energy-efficient Neural Network Inference on Edge Devicescited by 5
  8. 2024Edge AI: Evaluation of Model Compression Techniques for Convolutional Neural Networkscited by 4
  9. 2019EdgeAI: A Vision for Deep Learning in IoT Eracited by 2
  10. 2025QuantuneV2: Compiler-Based Local Metric-Driven Mixed Precision Quantization for Practical Embedded AI Applicationscited by 2
  11. 2023Dynamic Shuffle: An Efficient Channel Mixture Methodcited by 1
  12. 2023USDC: Unified Static and Dynamic Compression for Visual Transformercited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025On Accelerating Edge AI: Optimizing Resource-Constrained Environments
  2. 2022Resource-Constrained Edge AI with Early Exit Prediction
  3. 20213U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Acceleration
  4. 2025Constrained Edge AI Deployment: Fine-Tuning vs Distillation for LLM Compression
  5. 2024Dynamic Semantic Compression for CNN Inference in Multi-access Edge Computing: A Graph Reinforcement Learning-based Autoencoder
  6. 2025Hardware-Algorithm Co-Optimization of Early-Exit Neural Networks for Multi-Core Edge Accelerators
  7. 2022Graph Reinforcement Learning-based CNN Inference Offloading in Dynamic Edge Computing
  8. 2026Lightweight Transformer Architectures for Edge Devices in Real-Time Applications