# MotionDeltaCNN: Sparse CNN Inference of Frame Differences in Moving Camera Videos

arXiv: https://arxiv.org/abs/2210.09887 · HTML: https://mlsanity.com/paper/2210.09887/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 24 of 24)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Fixed Point Quantization of Deep Convolutional Networks](https://mlsanity.com/paper/1511.06393/) (2015)
- [Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Video](https://mlsanity.com/paper/1807.06667/) (2018)
- [Low-Latency Video Semantic Segmentation](https://mlsanity.com/paper/1804.00389/) (2018)
- [Towards High Performance Video Object Detection](https://mlsanity.com/paper/1711.11577/) (2017)
- [Clockwork Convnets for Video Semantic Segmentation](https://mlsanity.com/paper/1608.03609/) (2016)
- [Skip-Convolutions for Efficient Video Processing](https://mlsanity.com/paper/2104.11487/) (2021)
- [Dynamic Kernel Distillation for Efficient Pose Estimation in Videos](https://mlsanity.com/paper/1908.09216/) (2019)
- [Minimum Energy Quantized Neural Networks](https://mlsanity.com/paper/1711.00215/) (2017)
- [DeltaCNN: End-to-End CNN Inference of Sparse Frame Differences in Videos](https://mlsanity.com/paper/2203.03996/) (2022)
- [Recurrent Residual Module for Fast Inference in Videos](https://mlsanity.com/paper/1802.09723/) (2018)
- [Pixel-Level Bijective Matching for Video Object Segmentation](https://mlsanity.com/paper/2110.01644/) (2021)
- [Delta Distillation for Efficient Video Processing](https://mlsanity.com/paper/2203.09594/) (2022)
- [CBinfer: Exploiting Frame-to-Frame Locality for Faster Convolutional Network Inference on Video Streams](https://mlsanity.com/paper/1808.05488/) (2018)
- [Event Neural Networks](https://mlsanity.com/paper/2112.00891/) (2021)

## Built on by (most cited first, 5 of 5)

- [MaskVD: Region Masking for Efficient Video Object Detection](https://mlsanity.com/paper/2407.12067/) (2024)
- [D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reuse](https://mlsanity.com/paper/2506.14107/) (2025)
- [DeltaLLM: Delta-Key Sparse Attention for Edge LLMs](https://mlsanity.com/paper/2507.19608/) (2025)
- [Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensors](https://mlsanity.com/paper/2409.17341/) (2024)
- [ReFrame: Layer Caching for Accelerated Inference in Real-Time Rendering](https://mlsanity.com/paper/2506.13814/) (2025)

## Abstract (arXiv)

Convolutional neural network inference on video input is computationally expensive and requires high memory bandwidth. Recently, DeltaCNN managed to reduce the cost by only processing pixels with significant updates over the previous frame. However, DeltaCNN relies on static camera input. Moving cameras add new challenges in how to fuse newly unveiled image regions with already processed regions efficiently to minimize the update rate - without increasing memory overhead and without knowing the camera extrinsics of future frames. In this work, we propose MotionDeltaCNN, a sparse CNN inference framework that supports moving cameras. We introduce spherical buffers and padded convolutions to enable seamless fusion of newly unveiled regions and previously processed regions -- without increasing memory footprint. Our evaluation shows that we outperform DeltaCNN by up to 90% for moving camera videos.