# Class Based Thresholding in Early Exit Semantic Segmentation Networks

arXiv: https://arxiv.org/abs/2210.15621 · HTML: https://mlsanity.com/paper/2210.15621/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving.
- Builds on 23 of its 47 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Placed in

- Efficient LLM inference and serving / [Token pruning and merging for ViTs](https://mlsanity.com/inference/token-pruning-and-merging-for-vits/)

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Compute Trends Across Three Eras of Machine Learning](https://mlsanity.com/paper/2202.05924/) (2022)
- [Multiscale Deep Equilibrium Models](https://mlsanity.com/paper/2006.08656/) (2020)
- [Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentation](https://mlsanity.com/paper/2107.11264/) (2021)
- [Limitations of Neural Collapse for Understanding Generalization in Deep Learning](https://mlsanity.com/paper/2202.08384/) (2022)
- [CondenseNet V2: Sparse Feature Reactivation for Deep Networks](https://mlsanity.com/paper/2104.04382/) (2021)
- [Learn to Combine Modalities in Multimodal Deep Learning](https://mlsanity.com/paper/1805.11730/) (2018)
- [Anytime Dense Prediction with Confidence Adaptivity](https://mlsanity.com/paper/2104.00749/) (2021)
- [Multi-Exit Semantic Segmentation Networks](https://mlsanity.com/paper/2106.03527/) (2021)
- [Efficient Ladder-style DenseNets for Semantic Segmentation of Large Images](https://mlsanity.com/paper/1905.05661/) (2019)
- [Dynamically pruning segformer for efficient semantic segmentation](https://mlsanity.com/paper/2111.09499/) (2021)
- [E2CM: Early Exit via Class Means for Efficient Supervised and Unsupervised Learning](https://mlsanity.com/paper/2103.01148/) (2021)
- [Pruning Early Exit Networks](https://mlsanity.com/paper/2207.03644/) (2022)

## Built on by (most cited first, 5 of 5)

- [A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusion](https://mlsanity.com/paper/2501.07451/) (2025)
- [Memorization Capacity of Neural Networks with Conditional Computation](https://mlsanity.com/paper/2303.11247/) (2023)
- [On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance](https://mlsanity.com/paper/2403.17154/) (2024)
- [Gated Compression Layers for Efficient Always-On Models](https://mlsanity.com/paper/2303.08970/) (2023)
- [Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exit](https://mlsanity.com/paper/2609.12337/) (2026)

## Abstract (arXiv)

We propose Class Based Thresholding (CBT) to reduce the computational cost of early exit semantic segmentation models while preserving the mean intersection over union (mIoU) performance. A key idea of CBT is to exploit the naturally-occurring neural collapse phenomenon. Specifically, by calculating the mean prediction probabilities of each class in the training set, CBT assigns different masking threshold values to each class, so that the computation can be terminated sooner for pixels belonging to easy-to-predict classes. We show the effectiveness of CBT on Cityscapes and ADE20K datasets. CBT can reduce the computational cost by 23% compared to the previous state-of-the-art early exit models.