# Temporal Early Exits for Efficient Video Object Detection

arXiv: https://arxiv.org/abs/2106.11208 · HTML: https://mlsanity.com/paper/2106.11208/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Speed/accuracy trade-offs for modern convolutional object detectors](https://mlsanity.com/paper/1611.10012/) (2016)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Salient Object Detection: A Benchmark](https://mlsanity.com/paper/1501.02741/) (2015)
- [Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognition](https://mlsanity.com/paper/1509.08971/) (2015)
- [Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning](https://mlsanity.com/paper/1803.05268/) (2018)
- [Progressive Sparse Local Attention for Video object detection](https://mlsanity.com/paper/1903.09126/) (2019)

## Built on by (most cited first, 4 of 4)

- [Multi-Exit Vision Transformer for Dynamic Inference](https://mlsanity.com/paper/2106.15183/) (2021)
- [Early Time Classification with Accumulated Accuracy Gap Control](https://mlsanity.com/paper/2402.00857/) (2024)
- [NACHOS: Neural Architecture Search for Hardware Constrained Early Exit Neural Networks](https://mlsanity.com/paper/2401.13330/) (2024)
- [BlankSkip: Early-exit Object Detection onboard Nano-drones](https://mlsanity.com/paper/2603.28149/) (2026)

## Abstract (arXiv)

Transferring image-based object detectors to the domain of video remains challenging under resource constraints. Previous efforts utilised optical flow to allow unchanged features to be propagated, however, the overhead is considerable when working with very slowly changing scenes from applications such as surveillance. In this paper, we propose temporal early exits to reduce the computational complexity of per-frame video object detection. Multiple temporal early exit modules with low computational overhead are inserted at early layers of the backbone network to identify the semantic differences between consecutive frames. Full computation is only required if the frame is identified as having a semantic change to previous frames; otherwise, detection results from previous frames are reused. Experiments on CDnet show that our method significantly reduces the computational complexity and execution of per-frame video object detection up to 34 × compared to existing methods with an acceptable reduction of 2.2% in mAP.