# MVFlow: Deep Optical Flow Estimation of Compressed Videos with Motion Vector Prior

arXiv: https://arxiv.org/abs/2308.01568 · HTML: https://mlsanity.com/paper/2308.01568/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 3 Aug 2023.

## Builds on (most cited first, 24 of 24)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [Real-Time Intermediate Flow Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2011.06294/) (2020)
- [Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation](https://mlsanity.com/paper/1904.05290/) (2019)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [Deep Flow-Guided Video Inpainting](https://mlsanity.com/paper/1905.02884/) (2019)
- [Compressed Video Action Recognition](https://mlsanity.com/paper/1712.00636/) (2017)
- [Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement](https://mlsanity.com/paper/2003.01966/) (2020)
- [CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flow](https://mlsanity.com/paper/2203.16896/) (2022)
- [Global Matching with Overlapping Attention for Optical Flow Estimation](https://mlsanity.com/paper/2203.11335/) (2022)
- [Improving Deep Video Compression by Resolution-adaptive Flow Coding](https://mlsanity.com/paper/2009.05982/) (2020)
- [SKFlow: Learning Optical Flow with Super Kernels](https://mlsanity.com/paper/2205.14623/) (2022)
- [Optical Flow Estimation from a Single Motion-blurred Image](https://mlsanity.com/paper/2103.02996/) (2021)
- [Accelerating Video Object Segmentation with Compressed Video](https://mlsanity.com/paper/2107.12192/) (2021)

## Built on by (most cited first, 7 of 7)

- [Autogenic Language Embedding for Coherent Point Tracking](https://mlsanity.com/paper/2407.20730/) (2024)
- [CodingHomo: Bootstrapping Deep Homography With Video Coding](https://mlsanity.com/paper/2504.12165/) (2025)
- [Coding-Prior Guided Diffusion Network for Video Deblurring](https://mlsanity.com/paper/2504.12222/) (2025)
- [VOCA: Visual Odometry with Codec Awareness](https://mlsanity.com/paper/2607.00189/) (2026)
- [ANVIL: Accelerator-Native Video Interpolation via Codec Motion Vector Priors](https://mlsanity.com/paper/2603.26835/) (2026)
- [TERDNet: Transformer Encoder-Recurrent Decoder Network for Scene Change Detection](https://mlsanity.com/paper/2605.20822/) (2026)
- [MVTrack: Ultrafast Appearance-Free Moving Object Tracking from Compressed Bitstreams](https://mlsanity.com/paper/2608.10790/) (2026)

## Abstract (arXiv)

In recent years, many deep learning-based methods have been proposed to tackle the problem of optical flow estimation and achieved promising results. However, they hardly consider that most videos are compressed and thus ignore the pre-computed information in compressed video streams. Motion vectors, one of the compression information, record the motion of the video frames. They can be directly extracted from the compression code stream without computational cost and serve as a solid prior for optical flow estimation. Therefore, we propose an optical flow model, MVFlow, which uses motion vectors to improve the speed and accuracy of optical flow estimation for compressed videos. In detail, MVFlow includes a key Motion-Vector Converting Module, which ensures that the motion vectors can be transformed into the same domain of optical flow and then be utilized fully by the flow estimation module. Meanwhile, we construct four optical flow datasets for compressed videos containing frames and motion vectors in pairs. The experimental results demonstrate the superiority of our proposed MVFlow, which can reduce the AEPE by 1.09 compared to existing models or save 52% time to achieve similar accuracy to existing models.