# DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition

arXiv: https://arxiv.org/abs/1901.03460 · HTML: https://mlsanity.com/paper/1901.03460/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2019.

## Built on by (most cited first, 30 of 36)

- [VideoComposer: Compositional Video Synthesis with Motion Controllability](https://mlsanity.com/paper/2306.02018/) (2023)
- [MM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognition](https://mlsanity.com/paper/2108.09322/) (2021)
- [You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos](https://mlsanity.com/paper/2303.07863/) (2023)
- [Self-supervised Video Representation Learning by Context and Motion Decoupling](https://mlsanity.com/paper/2104.00862/) (2021)
- [Multi-Attention Network for Compressed Video Referring Object Segmentation](https://mlsanity.com/paper/2207.12622/) (2022)
- [A Coding Framework and Benchmark towards Low-Bitrate Video Understanding](https://mlsanity.com/paper/2202.02813/) (2022)
- [End-to-End Compressed Video Representation Learning for Generic Event Boundary Detection](https://mlsanity.com/paper/2203.15336/) (2022)
- [Gate-Shift-Fuse for Video Action Recognition](https://mlsanity.com/paper/2203.08897/) (2022)
- [Deep Unsupervised Key Frame Extraction for Efficient Video Classification](https://mlsanity.com/paper/2211.06742/) (2022)
- [SOAR: Scene-debiasing Open-set Action Recognition](https://mlsanity.com/paper/2309.01265/) (2023)
- [SMC++: Masked Learning of Unsupervised Video Semantic Compression](https://mlsanity.com/paper/2406.04765/) (2024)
- [CoPE-VideoLM: Leveraging Codec Primitives For Efficient Video Language Modeling](https://mlsanity.com/paper/2602.13191/) (2026)
- [Identity-aware Facial Expression Recognition in Compressed Video](https://mlsanity.com/paper/2101.00317/) (2021)
- [Compressed Vision for Efficient Video Understanding](https://mlsanity.com/paper/2210.02995/) (2022)
- [Video Event Extraction via Tracking Visual States of Arguments](https://mlsanity.com/paper/2211.01781/) (2022)
- [Efficient Semantic Segmentation by Altering Resolutions for Compressed Videos](https://mlsanity.com/paper/2303.07224/) (2023)
- [ReMoRa: Multimodal Large Language Model based on Refined Motion Representation for Long-Video Understanding](https://mlsanity.com/paper/2602.16412/) (2026)
- [Semantically Video Coding: Instill Static-Dynamic Clues into Structured Bitstream for AI Tasks](https://mlsanity.com/paper/2201.10162/) (2022)
- [AdaCodec: A Predictive Visual Code for Video MLLMs](https://mlsanity.com/paper/2606.02569/) (2026)
- [Task-Aware Encoder Control for Deep Video Compression](https://mlsanity.com/paper/2404.04848/) (2024)
- [TEAM-Net: Multi-modal Learning for Video Action Recognition with Partial Decoding](https://mlsanity.com/paper/2110.08814/) (2021)
- [Local Compressed Video Stream Learning for Generic Event Boundary Detection](https://mlsanity.com/paper/2309.15431/) (2023)
- [Unsupervised Motion Representation Enhanced Network for Action Recognition](https://mlsanity.com/paper/2103.03465/) (2021)
- [Towards Scalable Modeling of Compressed Videos for Efficient Action Recognition](https://mlsanity.com/paper/2503.13724/) (2025)
- [Video Captioning in Compressed Video](https://mlsanity.com/paper/2101.00359/) (2021)
- [Spatio-Temporal Video Representation Learning for AI Based Video Playback Style Prediction](https://mlsanity.com/paper/2110.01015/) (2021)
- [Representation Learning for Compressed Video Action Recognition via Attentive Cross-modal Interaction with Motion Enhancement](https://mlsanity.com/paper/2205.03569/) (2022)
- [Advancing Compressed Video Action Recognition through Progressive Knowledge Distillation](https://mlsanity.com/paper/2407.02713/) (2024)
- [ComPrivDet: Efficient Privacy Object Detection in Compressed Domains Through Inference Reuse](https://mlsanity.com/paper/2604.03640/) (2026)
- [Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit](https://mlsanity.com/paper/2606.31004/) (2026)

## Abstract (arXiv)

Motion has shown to be useful for video understanding, where motion is typically represented by optical flow. However, computing flow from video frames is very time-consuming. Recent works directly leverage the motion vectors and residuals readily available in the compressed video to represent motion at no cost. While this avoids flow computation, it also hurts accuracy since the motion vector is noisy and has substantially reduced resolution, which makes it a less discriminative motion representation. To remedy these issues, we propose a lightweight generator network, which reduces noises in motion vectors and captures fine motion details, achieving a more Discriminative Motion Cue (DMC) representation. Since optical flow is a more accurate motion representation, we train the DMC generator to approximate flow using a reconstruction loss and a generative adversarial loss, jointly with the downstream action classification task. Extensive evaluations on three action recognition benchmarks (HMDB-51, UCF-101, and a subset of Kinetics) confirm the effectiveness of our method. Our full system, consisting of the generator and the classifier, is coined as DMC-Net which obtains high accuracy close to that of using flow and runs two orders of magnitude faster than using optical flow at inference time.