# Efficient Video Semantic Segmentation with Labels Propagation and Refinement

arXiv: https://arxiv.org/abs/1912.11844 · HTML: https://mlsanity.com/paper/1912.11844/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Dec 2019.

## Built on by (most cited first, 18 of 18)

- [Mask Propagation for Efficient Video Semantic Segmentation](https://mlsanity.com/paper/2310.18954/) (2023)
- [Local Memory Attention for Fast Video Semantic Segmentation](https://mlsanity.com/paper/2101.01715/) (2021)
- [Mining Relations among Cross-Frame Affinities for Video Semantic Segmentation](https://mlsanity.com/paper/2207.10436/) (2022)
- [QDFormer: Towards Robust Audiovisual Segmentation in Complex Environments with Quantization-based Semantic Decomposition](https://mlsanity.com/paper/2310.00132/) (2023)
- [Event-guided Low-light Video Semantic Segmentation](https://mlsanity.com/paper/2411.00639/) (2024)
- [Accelerating Video Object Segmentation with Compressed Video](https://mlsanity.com/paper/2107.12192/) (2021)
- [BlockCopy: High-Resolution Video Processing with Block-Sparse Feature Propagation and Online Policies](https://mlsanity.com/paper/2108.09376/) (2021)
- [Efficient Semantic Segmentation by Altering Resolutions for Compressed Videos](https://mlsanity.com/paper/2303.07224/) (2023)
- [Warp-Refine Propagation: Semi-Supervised Auto-labeling via Cycle-consistency](https://mlsanity.com/paper/2109.13432/) (2021)
- [Evaluating SAM2 for Video Semantic Segmentation](https://mlsanity.com/paper/2512.01774/) (2025)
- [DNN-Driven Compressive Offloading for Edge-Assisted Semantic Video Segmentation](https://mlsanity.com/paper/2203.14481/) (2022)
- [Bootstrapped Self-Supervised Training with Monocular Video for Semantic Segmentation and Depth Estimation](https://mlsanity.com/paper/2103.11031/) (2021)
- [A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design](https://mlsanity.com/paper/2303.04315/) (2023)
- [A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects](https://mlsanity.com/paper/2506.13552/) (2025)
- [MED-VT++: Unifying Multimodal Learning with a Multiscale Encoder-Decoder Video Transformer](https://mlsanity.com/paper/2304.05930/) (2023)
- [RS-SSM: Refining Forgotten Specifics in State Space Model for Video Semantic Segmentation](https://mlsanity.com/paper/2603.24295/) (2026)
- [Point Cloud Segmentation Using Sparse Temporal Local Attention](https://mlsanity.com/paper/2112.00289/) (2021)
- [Co-SemDepth: Fast Joint Semantic Segmentation and Depth Estimation on Aerial Images](https://mlsanity.com/paper/2503.17982/) (2025)

## Abstract (arXiv)

This paper tackles the problem of real-time semantic segmentation of high definition videos using a hybrid GPU / CPU approach. We propose an Efficient Video Segmentation(EVS) pipeline that combines: (i) On the CPU, a very fast optical flow method, that is used to exploit the temporal aspect of the video and propagate semantic information from one frame to the next. It runs in parallel with the GPU. (ii) On the GPU, two Convolutional Neural Networks: A main segmentation network that is used to predict dense semantic labels from scratch, and a Refiner that is designed to improve predictions from previous frames with the help of a fast Inconsistencies Attention Module (IAM). The latter can identify regions that cannot be propagated accurately. We suggest several operating points depending on the desired frame rate and accuracy. Our pipeline achieves accuracy levels competitive to the existing real-time methods for semantic image segmentation(mIoU above 60%), while achieving much higher frame rates. On the popular Cityscapes dataset with high resolution frames (2048 x 1024), the proposed operating points range from 80 to 1000 Hz on a single GPU and CPU.