# Mimic The Raw Domain: Accelerating Action Recognition in the Compressed Domain

arXiv: https://arxiv.org/abs/1911.08206 · HTML: https://mlsanity.com/paper/1911.08206/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2019.

## Built on by (most cited first, 8 of 8)

- [Efficient Semantic Segmentation by Altering Resolutions for Compressed Videos](https://mlsanity.com/paper/2303.07224/) (2023)
- [ReMoRa: Multimodal Large Language Model based on Refined Motion Representation for Long-Video Understanding](https://mlsanity.com/paper/2602.16412/) (2026)
- [TEAM-Net: Multi-modal Learning for Video Action Recognition with Partial Decoding](https://mlsanity.com/paper/2110.08814/) (2021)
- [Towards Scalable Modeling of Compressed Videos for Efficient Action Recognition](https://mlsanity.com/paper/2503.13724/) (2025)
- [Representation Learning for Compressed Video Action Recognition via Attentive Cross-modal Interaction with Motion Enhancement](https://mlsanity.com/paper/2205.03569/) (2022)
- [Advancing Compressed Video Action Recognition through Progressive Knowledge Distillation](https://mlsanity.com/paper/2407.02713/) (2024)
- [GTM: Gray Temporal Model for Video Recognition](https://mlsanity.com/paper/2110.10348/) (2021)
- [Speeding Up Action Recognition Using Dynamic Accumulation of Residuals in Compressed Domain](https://mlsanity.com/paper/2209.14757/) (2022)

## Abstract (arXiv)

Video understanding usually requires expensive computation that prohibits its deployment, yet videos contain significant spatiotemporal redundancy that can be exploited. In particular, operating directly on the motion vectors and residuals in the compressed video domain can significantly accelerate the compute, by not using the raw videos which demand colossal storage capacity. Existing methods approach this task as a multiple modalities problem. In this paper we are approaching the task in a completely different way; we are looking at the data from the compressed stream as a one unit clip and propose that the residual frames can replace the original RGB frames from the raw domain. Furthermore, we are using teacher-student method to aid the network in the compressed domain to mimic the teacher network in the raw domain. We show experiments on three leading datasets (HMDB51, UCF1, and Kinetics) that approach state-of-the-art accuracy on raw video data by using compressed data. Our model MFCD-Net outperforms prior methods in the compressed domain and more importantly, our model has 11X fewer parameters and 3X fewer Flops, dramatically improving the efficiency of video recognition inference. This approach enables applying neural networks exclusively in the compressed domain without compromising accuracy while accelerating performance.