# Early-Exit with Class Exclusion for Efficient Inference of Neural Networks

arXiv: https://arxiv.org/abs/2309.13443 · HTML: https://mlsanity.com/paper/2309.13443/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving.
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 23 Sep 2023.

## Placed in

- Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/)

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Improved Techniques for Training Adaptive Deep Networks](https://mlsanity.com/paper/1908.06294/) (2019)
- [Learning to Weight Samples for Dynamic Early-exiting Networks](https://mlsanity.com/paper/2209.08310/) (2022)
- [Adaptive Loss-aware Quantization for Multi-bit Networks](https://mlsanity.com/paper/1912.08883/) (2019)
- [Class-based Quantization for Neural Networks](https://mlsanity.com/paper/2211.14928/) (2022)
- [PowerPruning: Selecting Weights and Activations for Power-Efficient Neural Network Acceleration](https://mlsanity.com/paper/2303.13997/) (2023)
- [SteppingNet: A Stepping Neural Network with Incremental Accuracy Enhancement](https://mlsanity.com/paper/2211.14926/) (2022)

## Built on by (most cited first, 8 of 8)

- [Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression](https://mlsanity.com/paper/2410.03765/) (2024)
- [LiveMind: Low-latency Large Language Models with Simultaneous Inference](https://mlsanity.com/paper/2406.14319/) (2024)
- [HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Serving](https://mlsanity.com/paper/2504.10724/) (2025)
- [EncodingNet: A Novel Encoding-based MAC Design for Efficient Neural Network Acceleration](https://mlsanity.com/paper/2402.18595/) (2024)
- [Attention Consistency Regularization for Interpretable Early-Exit Neural Networks](https://mlsanity.com/paper/2601.08891/) (2026)
- [Late Breaking Results: Conversion of Neural Networks into Logic Flows for Edge Computing](https://mlsanity.com/paper/2601.22151/) (2026)
- [DART: Input-Difficulty-AwaRe Adaptive Threshold for Early-Exit DNNs](https://mlsanity.com/paper/2603.12269/) (2026)
- [DiffLUT-Net: Differentiable Training of FPGA LUT Networks with Learnable Connectivity](https://mlsanity.com/paper/2609.09254/) (2026)

## Abstract (arXiv)

Deep neural networks (DNNs) have been successfully applied in various fields. In DNNs, a large number of multiply-accumulate (MAC) operations are required to be performed, posing critical challenges in applying them in resource-constrained platforms, e.g., edge devices. To address this challenge, in this paper, we propose a class-based early-exit for dynamic inference. Instead of pushing DNNs to make a dynamic decision at intermediate layers, we take advantage of the learned features in these layers to exclude as many irrelevant classes as possible, so that later layers only have to determine the target class among the remaining classes. When only one class remains at a layer, this class is the corresponding classification result. Experimental results demonstrate the computational cost of DNNs in inference can be reduced significantly with the proposed early-exit technique. The codes can be found at https://github.com/HWAI-TUDa/EarlyClassExclusion.