# Deep Spatio-Temporal Random Fields for Efficient Video Segmentation

arXiv: https://arxiv.org/abs/1807.03148 · HTML: https://mlsanity.com/paper/1807.03148/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2018.

## Built on by (most cited first, 15 of 15)

- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes](https://mlsanity.com/paper/2101.06085/) (2021)
- [Generative Adversarial Network for Future Hand Segmentation from Egocentric Video](https://mlsanity.com/paper/2203.11305/) (2022)
- [Rethinking Dilated Convolution for Real-time Semantic Segmentation](https://mlsanity.com/paper/2111.09957/) (2021)
- [RankSeg: Adaptive Pixel Classification with Image Category Ranking for Segmentation](https://mlsanity.com/paper/2203.04187/) (2022)
- [Global Aggregation then Local Distribution for Scene Parsing](https://mlsanity.com/paper/2107.13154/) (2021)
- [SERNet-Former: Semantic Segmentation by Efficient Residual Network with Attention-Boosting Gates and Attention-Fusion Networks](https://mlsanity.com/paper/2401.15741/) (2024)
- [Domain Adaptive Video Semantic Segmentation via Cross-Domain Moving Object Mixing](https://mlsanity.com/paper/2211.02307/) (2022)
- [A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects](https://mlsanity.com/paper/2506.13552/) (2025)
- [Efficient Video Segmentation Models with Per-frame Inference](https://mlsanity.com/paper/2202.12427/) (2022)
- [Dynamic Dual Sampling Module for Fine-Grained Semantic Segmentation](https://mlsanity.com/paper/2105.11657/) (2021)
- [Deep Learning Model with GA based Feature Selection and Context Integration](https://mlsanity.com/paper/2204.06189/) (2022)
- [Motion-Corrected Moving Average: Including Post-Hoc Temporal Information for Improved Video Segmentation](https://mlsanity.com/paper/2403.03120/) (2024)
- [Breaking The Ice: Video Segmentation for Close-Range Ice-Covered Waters](https://mlsanity.com/paper/2411.05225/) (2024)
- [Generalized Event Partonomy Inference with Structured Hierarchical Predictive Learning](https://mlsanity.com/paper/2512.04219/) (2025)

## Abstract (arXiv)

In this work we introduce a time- and memory-efficient method for structured prediction that couples neuron decisions across both space at time. We show that we are able to perform exact and efficient inference on a densely connected spatio-temporal graph by capitalizing on recent advances on deep Gaussian Conditional Random Fields (GCRFs). Our method, called VideoGCRF is (a) efficient, (b) has a unique global minimum, and (c) can be trained end-to-end alongside contemporary deep networks for video understanding. We experiment with multiple connectivity patterns in the temporal domain, and present empirical improvements over strong baselines on the tasks of both semantic and instance segmentation of videos.