Improving the Accuracy of Early Exits in Multi-Exit Architectures via Curriculum Learning
Not on the map. Why?
Abstract (arXiv)
Deploying deep learning services for time-sensitive and resource-constrained settings such as IoT using edge computing systems is a challenging task that requires dynamic adjustment of inference time. Multi-exit architectures allow deep neural networks to terminate their execution early in order to adhere to tight deadlines at the cost of accuracy. To mitigate this cost, in this paper we introduce a novel method called Multi-Exit Curriculum Learning that utilizes curriculum learning, a training strategy for neural networks that imitates human learning by sorting the training samples based on their difficulty and gradually introducing them to the network. Experiments on CIFAR-10 and CIFAR-100 datasets and various configurations of multi-exit architectures show that our method consistently improves the accuracy of early exits compared to the standard training approach.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 16 of its 28 references that are in the map. First posted on arXiv 21 Apr 2021.
Builds on
16 of 28 references.
References · 16
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2016Pruning Filters for Efficient ConvNetscited by 1,129
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
- 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
- 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
- 2019Convergence of Edge Computing and Deep Learning: A Comprehensive Surveycited by 120
4 more
- 2020Why should we add early exits to neural networks?cited by 70
- 2020Triple Wins: Boosting Accuracy, Robustness and Efficiency Together by Enabling Input-Adaptive Inferencecited by 45
- 2017Exploring the Regularity of Sparse Structure in Convolutional Neural Networkscited by 37
- 2020A Survey on Curriculum Learningcited by 15
Built on by
Citing papers per year: 2021: 2; 2022: 4; 2023: 0; 2024: 3.
9 papers.
- 2021Multi-Exit Vision Transformer for Dynamic Inferencecited by 28
- 2021Single-Layer Vision Transformers for More Accurate Early Exits with Less Overheadcited by 26
- 2024EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Modelscited by 13
- 2024Conditional computation in neural networks: principles and research trendscited by 10
- 2022Efficient High-Resolution Deep Learning: A Surveycited by 9
- 2022Graph Reinforcement Learning-based CNN Inference Offloading in Dynamic Edge Computingcited by 4
- 2024Dynamic Semantic Compression for CNN Inference in Multi-access Edge Computing: A Graph Reinforcement Learning-based Autoencodercited by 4
- 2022Crowd Counting on Heavily Compressed Images with Curriculum Pre-Trainingcited by 1
- 2022QuickNets: Saving Training and Preventing Overconfidence in Early-Exit Neural Architecturescited by 1
- 2024Conditional computation in neural networks: principles and research trends12 Mar 2024
- 2024EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models1 Feb 2024
- 2024Dynamic Semantic Compression for CNN Inference in Multi-access Edge Computing: A Graph Reinforcement Learning-based Autoencoder19 Jan 2024
- 2022QuickNets: Saving Training and Preventing Overconfidence in Early-Exit Neural Architectures25 Dec 2022
- 2022Graph Reinforcement Learning-based CNN Inference Offloading in Dynamic Edge Computing24 Oct 2022
- 2022Crowd Counting on Heavily Compressed Images with Curriculum Pre-Training15 Aug 2022
- 2022Efficient High-Resolution Deep Learning: A Survey26 Jul 2022
- 2021Multi-Exit Vision Transformer for Dynamic Inference29 Jun 2021
- 2021Single-Layer Vision Transformers for More Accurate Early Exits with Less Overhead19 May 2021
Similar, not cited
Close in wording, no citation either way
- 2021Consistency Training of Multi-exit Architectures for Sensor Data
- 2023Hierarchical Training of Deep Neural Networks Using Early Exiting
- 2022Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference
- 2025AEBNAS: Strengthening Exit Branches in Early-Exit Networks through Hardware-Aware Neural Architecture Search
- 2022Resource-Constrained Edge AI with Early Exit Prediction
- 2023Early-Exit with Class Exclusion for Efficient Inference of Neural Networks
- 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges
- 2025Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliability