# Fast Object Detection in Compressed Video

arXiv: https://arxiv.org/abs/1811.11057 · HTML: https://mlsanity.com/paper/1811.11057/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2018.

## Built on by (most cited first, 30 of 33)

- [You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos](https://mlsanity.com/paper/2303.07863/) (2023)
- [Self-supervised Video Representation Learning by Context and Motion Decoupling](https://mlsanity.com/paper/2104.00862/) (2021)
- [MMFuser: Multimodal Multi-Layer Feature Fuser for Fine-Grained Vision-Language Understanding](https://mlsanity.com/paper/2410.11829/) (2024)
- [A Coding Framework and Benchmark towards Low-Bitrate Video Understanding](https://mlsanity.com/paper/2202.02813/) (2022)
- [End-to-End Compressed Video Representation Learning for Generic Event Boundary Detection](https://mlsanity.com/paper/2203.15336/) (2022)
- [QueryProp: Object Query Propagation for High-Performance Video Object Detection](https://mlsanity.com/paper/2207.10959/) (2022)
- [SMC++: Masked Learning of Unsupervised Video Semantic Compression](https://mlsanity.com/paper/2406.04765/) (2024)
- [Boggart: Towards General-Purpose Acceleration of Retrospective Video Analytics](https://mlsanity.com/paper/2106.15315/) (2021)
- [Accelerating Video Object Segmentation with Compressed Video](https://mlsanity.com/paper/2107.12192/) (2021)
- [Efficient High-Resolution Deep Learning: A Survey](https://mlsanity.com/paper/2207.13050/) (2022)
- [Efficient Semantic Segmentation by Altering Resolutions for Compressed Videos](https://mlsanity.com/paper/2303.07224/) (2023)
- [AdaCodec: A Predictive Visual Code for Video MLLMs](https://mlsanity.com/paper/2606.02569/) (2026)
- [Real-time Online Multi-Object Tracking in Compressed Domain](https://mlsanity.com/paper/2204.02081/) (2022)
- [CoVA: Exploiting Compressed-Domain Analysis to Accelerate Video Analytics](https://mlsanity.com/paper/2207.00588/) (2022)
- [Task-Aware Encoder Control for Deep Video Compression](https://mlsanity.com/paper/2404.04848/) (2024)
- [PatchNet -- Short-range Template Matching for Efficient Video Processing](https://mlsanity.com/paper/2103.07371/) (2021)
- [Real-Time and Accurate Object Detection in Compressed Video by Long Short-term Feature Aggregation](https://mlsanity.com/paper/2103.14529/) (2021)
- [DFA: Dynamic Feature Aggregation for Efficient Video Object Detection](https://mlsanity.com/paper/2210.00588/) (2022)
- [Plug-and-Play Versatile Compressed Video Enhancement](https://mlsanity.com/paper/2504.15380/) (2025)
- [See Without Decoding: Motion-Vector-Based Tracking in Compressed Video](https://mlsanity.com/paper/2602.00153/) (2026)
- [Local Compressed Video Stream Learning for Generic Event Boundary Detection](https://mlsanity.com/paper/2309.15431/) (2023)
- [All-in-One Image Compression and Restoration](https://mlsanity.com/paper/2502.03649/) (2025)
- [MVP: Motion Vector Propagation for Zero-Shot Video Object Detection](https://mlsanity.com/paper/2509.18388/) (2025)
- [Video Captioning in Compressed Video](https://mlsanity.com/paper/2101.00359/) (2021)
- [CompSplat: Compression-aware 3D Gaussian Splatting for Real-world Video](https://mlsanity.com/paper/2602.09816/) (2026)
- [ComPrivDet: Efficient Privacy Object Detection in Compressed Domains Through Inference Reuse](https://mlsanity.com/paper/2604.03640/) (2026)
- [GTM: Gray Temporal Model for Video Recognition](https://mlsanity.com/paper/2110.10348/) (2021)
- [Speeding Up Action Recognition Using Dynamic Accumulation of Residuals in Compressed Domain](https://mlsanity.com/paper/2209.14757/) (2022)
- [Collaboratively Self-supervised Video Representation Learning for Action Recognition](https://mlsanity.com/paper/2401.07584/) (2024)
- [Integration of Communication and Computational Imaging](https://mlsanity.com/paper/2410.19415/) (2024)

## Abstract (arXiv)

Object detection in videos has drawn increasing attention since it is more practical in real scenarios. Most of the deep learning methods use CNNs to process each decoded frame in a video stream individually. However, the free of charge yet valuable motion information already embedded in the video compression format is usually overlooked. In this paper, we propose a fast object detection method by taking advantage of this with a novel Motion aided Memory Network (MMNet). The MMNet has two major advantages: 1) It significantly accelerates the procedure of feature extraction for compressed videos. It only need to run a complete recognition network for I-frames, i.e. a few reference frames in a video, and it produces the features for the following P frames (predictive frames) with a light weight memory network, which runs fast; 2) Unlike existing methods that establish an additional network to model motion of frames, we take full advantage of both motion vectors and residual errors that are freely available in video streams. To our best knowledge, the MMNet is the first work that investigates a deep convolutional detector on compressed videos. Our method is evaluated on the large-scale ImageNet VID dataset, and the results show that it is 3x times faster than single image detector R-FCN and 10x times faster than high-performance detector MANet at a minor accuracy loss.