# A Survey on Efficiency Optimization Techniques for DNN-based Video Analytics: Process Systems, Algorithms, and Applications

arXiv: https://arxiv.org/abs/2507.15628 · HTML: https://mlsanity.com/paper/2507.15628/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 52 of its 143 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Builds on (most cited first, 30 of 52)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [InternVideo: General Video Foundation Models via Generative and Discriminative Learning](https://mlsanity.com/paper/2212.03191/) (2022)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [Unmasked Teacher: Towards Training-Efficient Video Foundation Models](https://mlsanity.com/paper/2303.16058/) (2023)
- [OmniVL:One Foundation Model for Image-Language and Video-Language Tasks](https://mlsanity.com/paper/2209.07526/) (2022)
- [AdaFrame: Adaptive Frame Selection for Fast Video Recognition](https://mlsanity.com/paper/1811.12432/) (2018)
- [SCSampler: Sampling Salient Clips from Video for Efficient Action Recognition](https://mlsanity.com/paper/1904.04289/) (2019)
- [Pixel-Adaptive Convolutional Neural Networks](https://mlsanity.com/paper/1904.05373/) (2019)
- [A Review on Deep Learning Techniques for Video Prediction](https://mlsanity.com/paper/2004.05214/) (2020)
- [AR-Net: Adaptive Frame Resolution for Efficient Action Recognition](https://mlsanity.com/paper/2007.15796/) (2020)
- [Adaptive Focus for Efficient Video Recognition](https://mlsanity.com/paper/2105.03245/) (2021)
- [Efficient Video Super-Resolution through Recurrent Latent Space Propagation](https://mlsanity.com/paper/1909.08080/) (2019)
- [NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Vision](https://mlsanity.com/paper/1810.10090/) (2018)
- [Multi-Agent Reinforcement Learning Based Frame Sampling for Effective Untrimmed Video Recognition](https://mlsanity.com/paper/1907.13369/) (2019)
- [Online Model Distillation for Efficient Video Inference](https://mlsanity.com/paper/1812.02699/) (2018)
- [FrameExit: Conditional Early Exiting for Efficient Video Recognition](https://mlsanity.com/paper/2104.13400/) (2021)
- [Video Salient Object Detection via Fully Convolutional Networks](https://mlsanity.com/paper/1702.00871/) (2017)
- [Revisiting Temporal Modeling for Video Super-resolution](https://mlsanity.com/paper/2008.05765/) (2020)
- [Temporal RoI Align for Video Object Recognition](https://mlsanity.com/paper/2109.03495/) (2021)
- [BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics](https://mlsanity.com/paper/1805.01046/) (2018)
- [Distilled Semantics for Comprehensive Scene Understanding from Videos](https://mlsanity.com/paper/2003.14030/) (2020)
- [Scaling Video Analytics on Constrained Edge Nodes](https://mlsanity.com/paper/1905.13536/) (2019)
- [Dynamic Network Quantization for Efficient Video Inference](https://mlsanity.com/paper/2108.10394/) (2021)
- [Skip-Convolutions for Efficient Video Processing](https://mlsanity.com/paper/2104.11487/) (2021)

## Abstract (arXiv)

The explosive growth of video data in recent years has brought higher demands for video analytics, where accuracy and efficiency remain the two primary concerns. Deep neural networks (DNNs) have been widely adopted to ensure accuracy; however, improving their efficiency in video analytics remains an open challenge. Different from existing surveys that make summaries of DNN-based video mainly from the accuracy optimization aspect, in this survey, we aim to provide a thorough review of optimization techniques focusing on the improvement of the efficiency of DNNs in video analytics. We organize existing methods in a bottom-up manner, covering multiple perspectives such as hardware support, data processing, operational deployment, etc. Finally, based on the optimization framework and existing works, we analyze and discuss the problems and challenges in the performance optimization of DNN-based video analytics.