# SCAN: A Scalable Neural Networks Framework Towards Compact and Efficient Models

arXiv: https://arxiv.org/abs/1906.03951 · HTML: https://mlsanity.com/paper/1906.03951/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2019.

## Built on by (most cited first, 22 of 22)

- [LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding](https://mlsanity.com/paper/2404.16710/) (2024)
- [Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding](https://mlsanity.com/paper/2310.05424/) (2023)
- [Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation](https://mlsanity.com/paper/2203.06321/) (2022)
- [Auto-Split: A General Framework of Collaborative Edge-Cloud AI](https://mlsanity.com/paper/2108.13041/) (2021)
- [Contrastive Deep Supervision](https://mlsanity.com/paper/2207.05306/) (2022)
- [It's always personal: Using Early Exits for Efficient On-Device CNN Personalisation](https://mlsanity.com/paper/2102.01393/) (2021)
- [Multi-Exit Semantic Segmentation Networks](https://mlsanity.com/paper/2106.03527/) (2021)
- [Revisiting Model's Uncertainty and Confidences for Adversarial Example Detection](https://mlsanity.com/paper/2103.05354/) (2021)
- [Deep Neural Network-based Enhancement for Image and Video Streaming Systems: A Survey and Future Directions](https://mlsanity.com/paper/2106.03727/) (2021)
- [Compacting Deep Neural Networks for Internet of Things: Methods and Applications](https://mlsanity.com/paper/2103.11083/) (2021)
- [An Interpretive Constrained Linear Model for ResNet and MgNet](https://mlsanity.com/paper/2112.07441/) (2021)
- [Self-Contrastive Learning: Single-viewed Supervised Contrastive Framework using Sub-network](https://mlsanity.com/paper/2106.15499/) (2021)
- [FAN-Trans: Online Knowledge Distillation for Facial Action Unit Detection](https://mlsanity.com/paper/2211.06143/) (2022)
- [A Closer Look at Branch Classifiers of Multi-exit Architectures](https://mlsanity.com/paper/2204.13347/) (2022)
- [Multimodal Adaptive Inference for Document Image Classification with Anytime Early Exiting](https://mlsanity.com/paper/2405.12705/) (2024)
- [Deep Feature Surgery: Towards Accurate and Efficient Multi-Exit Networks](https://mlsanity.com/paper/2407.13986/) (2024)
- [SATBench: Benchmarking the speed-accuracy tradeoff in object recognition by humans and dynamic neural networks](https://mlsanity.com/paper/2206.08427/) (2022)
- [An Ensemble Mobile-Cloud Computing Method for Affordable and Accurate Glucometer Readout](https://mlsanity.com/paper/2301.01758/) (2023)
- [Revisiting Data Augmentation in Model Compression: An Empirical and Comprehensive Study](https://mlsanity.com/paper/2305.13232/) (2023)
- [Feature Noise Resilient for QoS Prediction with Probabilistic Deep Supervision](https://mlsanity.com/paper/2308.02580/) (2023)
- [FiRST: Finetuning Router-Selective Transformers for Input-Adaptive Latency Reduction](https://mlsanity.com/paper/2410.12513/) (2024)
- [Fusion Sampling Validation in Data Partitioning for Machine Learning](https://mlsanity.com/paper/2508.01325/) (2025)

## Abstract (arXiv)

Remarkable achievements have been attained by deep neural networks in various applications. However, the increasing depth and width of such models also lead to explosive growth in both storage and computation, which has restricted the deployment of deep neural networks on resource-limited edge devices. To address this problem, we propose the so-called SCAN framework for networks training and inference, which is orthogonal and complementary to existing acceleration and compression methods. The proposed SCAN firstly divides neural networks into multiple sections according to their depth and constructs shallow classifiers upon the intermediate features of different sections. Moreover, attention modules and knowledge distillation are utilized to enhance the accuracy of shallow classifiers. Based on this architecture, we further propose a threshold controlled scalable inference mechanism to approach human-like sample-specific inference. Experimental results show that SCAN can be easily equipped on various neural networks without any adjustment on hyper-parameters or neural networks architectures, yielding significant performance gain on CIFAR100 and ImageNet. Codes will be released on github soon.