# Energy-Aware Dynamic Neural Inference

arXiv: https://arxiv.org/abs/2411.02471 · HTML: https://mlsanity.com/paper/2411.02471/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 4 Nov 2024.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Revisiting the Calibration of Modern Neural Networks](https://mlsanity.com/paper/2106.07998/) (2021)
- [Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges](https://mlsanity.com/paper/2103.04505/) (2021)
- [Why should we add early exits to neural networks?](https://mlsanity.com/paper/2004.12814/) (2020)
- [A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking](https://mlsanity.com/paper/2309.02031/) (2023)
- [MobiSR: Efficient On-Device Super-Resolution through Heterogeneous Mobile Processors](https://mlsanity.com/paper/1908.07985/) (2019)
- [Intelligence Beyond the Edge: Inference on Intermittent Embedded Systems](https://mlsanity.com/paper/1810.07751/) (2018)
- [Intermittent Inference with Nonuniformly Compressed Multi-Exit Neural Network for Energy Harvesting Powered Devices](https://mlsanity.com/paper/2004.11293/) (2020)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [Towards Battery-Free Machine Learning and Inference in Underwater Environments](https://mlsanity.com/paper/2202.08174/) (2022)
- [A3CLNN: Spatial, Spectral and Multiscale Attention ConvLSTM Neural Network for Multisource Remote Sensing Data Classification](https://mlsanity.com/paper/2204.04462/) (2022)
- [Sensing, Computing, and Communication for Energy Harvesting IoTs: A Survey](https://mlsanity.com/paper/1905.03949/) (2019)
- [Sustainable Edge Intelligence Through Energy-Aware Early Exiting](https://mlsanity.com/paper/2305.14094/) (2023)

## Built on by (most cited first, 1 of 1)

- [FM-CAC: Carbon-Aware Control for Battery-Buffered Edge AI via Time-Series Foundation Models](https://mlsanity.com/paper/2604.16448/) (2026)

## Abstract (arXiv)

The growing demand for intelligent applications beyond the network edge, coupled with the need for sustainable operation, are driving the seamless integration of deep learning (DL) algorithms into energy-limited, and even energy-harvesting end-devices. However, the stochastic nature of ambient energy sources often results in insufficient harvesting rates, failing to meet the energy requirements for inference and causing significant performance degradation in energy-agnostic systems. To address this problem, we consider an on-device adaptive inference system equipped with an energy-harvester and finite-capacity energy storage. We then allow the device to reduce the run-time execution cost on-demand, by either switching between differently-sized neural networks, referred to as multi-model selection (MMS), or by enabling earlier predictions at intermediate layers, called early exiting (EE). The model to be employed, or the exit point is then dynamically chosen based on the energy storage and harvesting process states. We also study the efficacy of integrating the prediction confidence into the decision-making process. We derive a principled policy with theoretical guarantees for confidence-aware and -agnostic controllers. Moreover, in multi-exit networks, we study the advantages of taking decisions incrementally, exit-by-exit, by designing a lightweight reinforcement learning-based controller. Experimental results show that, as the rate of the ambient energy increases, energy- and confidence-aware control schemes show approximately 5% improvement in accuracy compared to their energy-aware confidence-agnostic counterparts. Incremental approaches achieve even higher accuracy, particularly when the energy storage capacity is limited relative to the energy consumption of the inference model.