# GSVNet: Guided Spatially-Varying Convolution for Fast Semantic Segmentation on Video

arXiv: https://arxiv.org/abs/2103.08834 · HTML: https://mlsanity.com/paper/2103.08834/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 16 references that are in the map.
- First posted on arXiv 16 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation](https://mlsanity.com/paper/1803.06815/) (2018)
- [Fast-SCNN: Fast Semantic Segmentation Network](https://mlsanity.com/paper/1902.04502/) (2019)
- [In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Images](https://mlsanity.com/paper/1903.08469/) (2019)
- [Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Video](https://mlsanity.com/paper/1807.06667/) (2018)
- [Temporally Distributed Networks for Fast Video Semantic Segmentation](https://mlsanity.com/paper/2004.01800/) (2020)
- [Low-Latency Video Semantic Segmentation](https://mlsanity.com/paper/1804.00389/) (2018)
- [Clockwork Convnets for Video Semantic Segmentation](https://mlsanity.com/paper/1608.03609/) (2016)
- [Dynamic Video Segmentation Network](https://mlsanity.com/paper/1804.00931/) (2018)
- [Fast Semantic Segmentation on Video Using Block Motion-Based Feature Interpolation](https://mlsanity.com/paper/1803.07742/) (2018)

## Built on by (most cited first, 11 of 11)

- [Mask Propagation for Efficient Video Semantic Segmentation](https://mlsanity.com/paper/2310.18954/) (2023)
- [Mining Relations among Cross-Frame Affinities for Video Semantic Segmentation](https://mlsanity.com/paper/2207.10436/) (2022)
- [Learning Local and Global Temporal Contexts for Video Semantic Segmentation](https://mlsanity.com/paper/2204.03330/) (2022)
- [Event-guided Low-light Video Semantic Segmentation](https://mlsanity.com/paper/2411.00639/) (2024)
- [Vanishing-Point-Guided Video Semantic Segmentation of Driving Scenes](https://mlsanity.com/paper/2401.15261/) (2024)
- [Bootstrapping Video Semantic Segmentation Model via Distillation-assisted Test-Time Adaptation](https://mlsanity.com/paper/2604.10950/) (2026)
- [A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design](https://mlsanity.com/paper/2303.04315/) (2023)
- [A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects](https://mlsanity.com/paper/2506.13552/) (2025)
- [RS-SSM: Refining Forgotten Specifics in State Space Model for Video Semantic Segmentation](https://mlsanity.com/paper/2603.24295/) (2026)
- [ICFRNet: Image Complexity Prior Guided Feature Refinement for Real-time Semantic Segmentation](https://mlsanity.com/paper/2408.13771/) (2024)
- [Can Unsupervised Segmentation Reduce Annotation Costs for Video Semantic Segmentation?](https://mlsanity.com/paper/2603.27697/) (2026)

## Abstract (arXiv)

This paper addresses fast semantic segmentation on video.Video segmentation often calls for real-time, or even fasterthan real-time, processing. One common recipe for conserving computation arising from feature extraction is to propagate features of few selected keyframes. However, recent advances in fast image segmentation make these solutions less attractive. To leverage fast image segmentation for furthering video segmentation, we propose a simple yet efficient propagation framework. Specifically, we perform lightweight flow estimation in 1/8-downscaled image space for temporal warping in segmentation outpace space. Moreover, we introduce a guided spatially-varying convolution for fusing segmentations derived from the previous and current frames, to mitigate propagation error and enable lightweight feature extraction on non-keyframes. Experimental results on Cityscapes and CamVid show that our scheme achieves the state-of-the-art accuracy-throughput trade-off on video segmentation.