# Accelerating Video Object Segmentation with Compressed Video

arXiv: https://arxiv.org/abs/2107.12192 · HTML: https://mlsanity.com/paper/2107.12192/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 26 Jul 2021.

## Builds on (most cited first, 26 of 26)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation](https://mlsanity.com/paper/2106.05210/) (2021)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Learning Video Object Segmentation from Static Images](https://mlsanity.com/paper/1612.02646/) (2016)
- [Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion](https://mlsanity.com/paper/2103.07941/) (2021)
- [Kernelized Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2007.08270/) (2020)
- [Online Adaptation of Convolutional Neural Networks for Video Object Segmentation](https://mlsanity.com/paper/1706.09364/) (2017)
- [Compressed Video Action Recognition](https://mlsanity.com/paper/1712.00636/) (2017)
- [Video Object Segmentation Without Temporal Information](https://mlsanity.com/paper/1709.06031/) (2017)
- [SegFlow: Joint Learning for Video Object Segmentation and Optical Flow](https://mlsanity.com/paper/1709.06750/) (2017)
- [PReMVOS: Proposal-generation, Refinement and Merging for Video Object Segmentation](https://mlsanity.com/paper/1807.09190/) (2018)
- [Collaborative Video Object Segmentation by Multi-Scale Foreground-Background Integration](https://mlsanity.com/paper/2010.06349/) (2020)
- [Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Video](https://mlsanity.com/paper/1807.06667/) (2018)
- [Learning Fast and Robust Target Models for Video Object Segmentation](https://mlsanity.com/paper/2003.00908/) (2020)
- [SwiftNet: Real-time Video Object Segmentation](https://mlsanity.com/paper/2102.04604/) (2021)
- [Low-Latency Video Semantic Segmentation](https://mlsanity.com/paper/1804.00389/) (2018)
- [A Transductive Approach for Video Object Segmentation](https://mlsanity.com/paper/2004.07193/) (2020)
- [Video Generation from Single Semantic Label Map](https://mlsanity.com/paper/1903.04480/) (2019)
- [State-Aware Tracker for Real-Time Video Object Segmentation](https://mlsanity.com/paper/2003.00482/) (2020)
- [Fast Object Detection in Compressed Video](https://mlsanity.com/paper/1811.11057/) (2018)
- [Efficient Video Semantic Segmentation with Labels Propagation and Refinement](https://mlsanity.com/paper/1912.11844/) (2019)

## Built on by (most cited first, 10 of 10)

- [Putting the Object Back into Video Object Segmentation](https://mlsanity.com/paper/2310.12982/) (2023)
- [You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos](https://mlsanity.com/paper/2303.07863/) (2023)
- [Look Before You Match: Instance Understanding Matters in Video Object Segmentation](https://mlsanity.com/paper/2212.06826/) (2022)
- [Mask Propagation for Efficient Video Semantic Segmentation](https://mlsanity.com/paper/2310.18954/) (2023)
- [Region Aware Video Object Segmentation with Deep Motion Modeling](https://mlsanity.com/paper/2207.10258/) (2022)
- [MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation](https://mlsanity.com/paper/2303.07815/) (2023)
- [MVFlow: Deep Optical Flow Estimation of Compressed Videos with Motion Vector Prior](https://mlsanity.com/paper/2308.01568/) (2023)
- [AdaCodec: A Predictive Visual Code for Video MLLMs](https://mlsanity.com/paper/2606.02569/) (2026)
- [Deep video representation learning: a survey](https://mlsanity.com/paper/2405.06574/) (2024)
- [MVTrack: Ultrafast Appearance-Free Moving Object Tracking from Compressed Bitstreams](https://mlsanity.com/paper/2608.10790/) (2026)

## Abstract (arXiv)

We propose an efficient plug-and-play acceleration framework for semi-supervised video object segmentation by exploiting the temporal redundancies in videos presented by the compressed bitstream. Specifically, we propose a motion vector-based warping method for propagating segmentation masks from keyframes to other frames in a bi-directional and multi-hop manner. Additionally, we introduce a residual-based correction module that can fix wrongly propagated segmentation masks from noisy or erroneous motion vectors. Our approach is flexible and can be added on top of several existing video object segmentation algorithms. We achieved highly competitive results on DAVIS17 and YouTube-VOS on various base models with substantial speed-ups of up to 3.5X with minor drops in accuracy.