# Energy-efficient Amortized Inference with Cascaded Deep Classifiers

arXiv: https://arxiv.org/abs/1710.03368 · HTML: https://mlsanity.com/paper/1710.03368/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2017.

## Built on by (most cited first, 15 of 15)

- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [Agreement-Based Cascading for Efficient Inference](https://mlsanity.com/paper/2407.02348/) (2024)
- [Boost Test-Time Performance with Closed-Loop Inference](https://mlsanity.com/paper/2203.10853/) (2022)
- [EREBA: Black-box Energy Testing of Adaptive Neural Networks](https://mlsanity.com/paper/2202.06084/) (2022)
- [Batching for Green AI -- An Exploratory Study on Inference](https://mlsanity.com/paper/2307.11434/) (2023)
- [SlowFormer: Universal Adversarial Patch for Attack on Compute and Energy Efficiency of Inference Efficient Vision Transformers](https://mlsanity.com/paper/2310.02544/) (2023)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [Perturbed Iterate SGD for Lipschitz Continuous Loss Functions with Numerical Error and Adaptive Step Sizes](https://mlsanity.com/paper/2211.04655/) (2022)
- [USDC: Unified Static and Dynamic Compression for Visual Transformer](https://mlsanity.com/paper/2310.11117/) (2023)
- [AEBNAS: Strengthening Exit Branches in Early-Exit Networks through Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2512.10671/) (2025)
- [KLAS: Using Similarity to Stitch Neural Networks for Improved Accuracy-Efficiency Tradeoffs](https://mlsanity.com/paper/2605.29259/) (2026)
- [Improving the Performance of DNN-based Software Services using Automated Layer Caching](https://mlsanity.com/paper/2209.08625/) (2022)
- [Puppet-CNN: Continuous Parameter Dynamics for Input-Adaptive Convolutional Networks](https://mlsanity.com/paper/2411.12876/) (2024)
- [ERDE: Entropy-Regularized Distillation for Early-exit](https://mlsanity.com/paper/2510.04856/) (2025)

## Abstract (arXiv)

Deep neural networks have been remarkable successful in various AI tasks but often cast high computation and energy cost for energy-constrained applications such as mobile sensing. We address this problem by proposing a novel framework that optimizes the prediction accuracy and energy cost simultaneously, thus enabling effective cost-accuracy trade-off at test time. In our framework, each data instance is pushed into a cascade of deep neural networks with increasing sizes, and a selection module is used to sequentially determine when a sufficiently accurate classifier can be used for this data instance. The cascade of neural networks and the selection module are jointly trained in an end-to-end fashion by the REINFORCE algorithm to optimize a trade-off between the computational cost and the predictive accuracy. Our method is able to simultaneously improve the accuracy and efficiency by learning to assign easy instances to fast yet sufficiently accurate classifiers to save computation and energy cost, while assigning harder instances to deeper and more powerful classifiers to ensure satisfiable accuracy. With extensive experiments on several image classification datasets using cascaded ResNet classifiers, we demonstrate that our method outperforms the standard well-trained ResNets in accuracy but only requires less than 20% and 50% FLOPs cost on the CIFAR-10/100 datasets and 66% on the ImageNet dataset, respectively.