# Temporal Decisions: Leveraging Temporal Correlation for Efficient Decisions in Early Exit Neural Networks

arXiv: https://arxiv.org/abs/2403.07958 · HTML: https://mlsanity.com/paper/2403.07958/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 10 of 10)

- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [BERT Loses Patience: Fast and Robust Inference with Early Exit](https://mlsanity.com/paper/2006.04152/) (2020)
- [Adaptive Neural Networks for Efficient Inference](https://mlsanity.com/paper/1702.07811/) (2017)
- [Hello Edge: Keyword Spotting on Microcontrollers](https://mlsanity.com/paper/1711.07128/) (2017)
- [Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognition](https://mlsanity.com/paper/1509.08971/) (2015)
- [Changing Model Behavior at Test-Time Using Reinforcement Learning](https://mlsanity.com/paper/1702.07780/) (2017)
- [Impatient DNNs - Deep Neural Networks with Dynamic Time Budgets](https://mlsanity.com/paper/1610.02850/) (2016)
- [Temporal Patience: Efficient Adaptive Deep Learning for Embedded Radar Data Processing](https://mlsanity.com/paper/2309.05686/) (2023)

## Built on by (most cited first, 1 of 1)

- [On-Demand Multi-Task Sparsity for Efficient Large-Model Deployment on Edge Devices](https://mlsanity.com/paper/2511.19986/) (2025)

## Abstract (arXiv)

Deep Learning is becoming increasingly relevant in Embedded and Internet-of-things applications. However, deploying models on embedded devices poses a challenge due to their resource limitations. This can impact the model's inference accuracy and latency. One potential solution are Early Exit Neural Networks, which adjust model depth dynamically through additional classifiers attached between their hidden layers. However, the real-time termination decision mechanism is critical for the system's efficiency, latency, and sustained accuracy. This paper introduces Difference Detection and Temporal Patience as decision mechanisms for Early Exit Neural Networks. They leverage the temporal correlation present in sensor data streams to efficiently terminate the inference. We evaluate their effectiveness in health monitoring, image classification, and wake-word detection tasks. Our novel contributions were able to reduce the computational footprint compared to established decision mechanisms significantly while maintaining higher accuracy scores. We achieved a reduction of mean operations per inference by up to 80% while maintaining accuracy levels within 5% of the original model. These findings highlight the importance of considering temporal correlation in sensor data to improve the termination decision.