mlsanityData to 14 Oct 2026

DeltaCNN: End-to-End CNN Inference of Sparse Frame Differences in Videos

arXiv 2203.03996 ↗First posted 8 Mar 2022cs.CV cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Convolutional neural network inference on video data requires powerful hardware for real-time processing. Given the inherent coherence across consecutive frames, large parts of a video typically change little. By skipping identical image regions and truncating insignificant pixel updates, computational redundancy can in theory be reduced significantly. However, these theoretical savings have been difficult to translate into practice, as sparse updates hamper computational consistency and memory access coherence; which are key for efficiency on real hardware. With DeltaCNN, we present a sparse convolutional neural network framework that enables sparse frame-by-frame updates to accelerate video inference in practice. We provide sparse implementations for all typical CNN layers and propagate sparse feature updates end-to-end - without accumulating errors over time. DeltaCNN is applicable to all convolutional neural networks without retraining. To the best of our knowledge, we are the first to significantly outperform the dense reference, cuDNN, in practical settings, achieving speedups of up to 7x with only marginal differences in accuracy.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 23 of its 35 references that are in the map. First posted on arXiv 8 Mar 2022.

Builds on

23 of 35 references.

References · 23

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  4. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  5. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,020
  6. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  7. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  8. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  9. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 295
  10. 2016Deep Feature Flow for Video Recognitioncited by 186
  11. 2015Fixed Point Quantization of Deep Convolutional Networkscited by 143
  12. 2020MOTChallenge: A Benchmark for Single-Camera Multiple Target Trackingcited by 136
11 more
  1. 2018Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Videocited by 78
  2. 2018Low-Latency Video Semantic Segmentationcited by 61
  3. 2017Towards High Performance Video Object Detectioncited by 60
  4. 2016Clockwork Convnets for Video Semantic Segmentationcited by 55
  5. 2018SBNet: Sparse Blocks Network for Fast Inferencecited by 54
  6. 2021Skip-Convolutions for Efficient Video Processingcited by 32
  7. 2019Dynamic Kernel Distillation for Efficient Pose Estimation in Videoscited by 30
  8. 2017Minimum Energy Quantized Neural Networkscited by 21
  9. 2018Recurrent Residual Module for Fast Inference in Videoscited by 13
  10. 2018CBinfer: Exploiting Frame-to-Frame Locality for Faster Convolutional Network Inference on Video Streamscited by 6
  11. 2018Pack and Detect: Fast Object Detection in Videos Using Region-of-Interest Packingcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 8; 2024: 4; 2025: 4.

18 papers.

Sort:

By area:

  1. 2022Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Modelscited by 45
  2. 2023Eventful Transformers: Leveraging Temporal Redundancy in Vision Transformerscited by 13
  3. 2022MotionDeltaCNN: Sparse CNN Inference of Frame Differences in Moving Camera Videoscited by 5
  4. 2024MaskVD: Region Masking for Efficient Video Object Detectioncited by 5
  5. 2025D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reusecited by 5
  6. 2025DeltaLLM: Delta-Key Sparse Attention for Edge LLMscited by 5
  7. 2023Accelerated Event-Based Feature Detection and Compression for Surveillance Video Systemscited by 3
  8. 2023ResQ: Residual Quantization for Video Perceptioncited by 2
  9. 2023Sparse and Privacy-enhanced Representation for Human Pose Estimationcited by 2
  10. 2024Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensorscited by 1
  11. 2023EvConv: Fast CNN Inference on Event Camera Inputs For High-Speed Robot Perceptioncited by 0
  12. 2023Incrementally-Computable Neural Networks: Efficient Inference for Dynamic Inputscited by 0
6 more
  1. 2023Uncertainty-Guided Spatial Pruning Architecture for Efficient Frame Interpolationcited by 0
  2. 2023Context-Aware Iteration Policy Network for Efficient Optical Flow Estimationcited by 0
  3. 2024Learning K-U-Net with constant complexity: An Application to time series forecastingcited by 0
  4. 2024SparseTem: Boosting the Efficiency of CNN-Based Video Encoders by Exploiting Temporal Continuitycited by 0
  5. 2025ReFrame: Layer Caching for Accelerated Inference in Real-Time Renderingcited by 0
  6. 2025evMLP: An Efficient Event-Driven MLP Architecture for Visioncited by 0