# Recurrent Fully Convolutional Networks for Video Segmentation

arXiv: https://arxiv.org/abs/1606.00487 · HTML: https://mlsanity.com/paper/1606.00487/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2016.

## Built on by (most cited first, 10 of 10)

- [Deep Learning Based 3D Segmentation: A Survey](https://mlsanity.com/paper/2103.05423/) (2021)
- [Exploiting Temporal State Space Sharing for Video Semantic Segmentation](https://mlsanity.com/paper/2503.20824/) (2025)
- [Rectify ViT Shortcut Learning by Visual Saliency](https://mlsanity.com/paper/2206.08567/) (2022)
- [PLGAN: Generative Adversarial Networks for Power-Line Segmentation in Aerial Images](https://mlsanity.com/paper/2204.07243/) (2022)
- [Self-Gated Memory Recurrent Network for Efficient Scalable HDR Deghosting](https://mlsanity.com/paper/2112.13050/) (2021)
- [Convex Hull Prediction for Adaptive Video Streaming by Recurrent Learning](https://mlsanity.com/paper/2206.04877/) (2022)
- [Efficient Frame Extraction: A Novel Approach Through Frame Similarity and Surgical Tool Tracking for Video Segmentation](https://mlsanity.com/paper/2501.11153/) (2025)
- [Detecting Slag Formations with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2110.06640/) (2021)
- [ItNet: iterative neural networks with small graphs for accurate, efficient and anytime semantic segmentation](https://mlsanity.com/paper/2101.08685/) (2021)
- [Shift-Memory Network for Temporal Scene Segmentation](https://mlsanity.com/paper/2202.08399/) (2022)

## Abstract (arXiv)

Image segmentation is an important step in most visual tasks. While convolutional neural networks have shown to perform well on single image segmentation, to our knowledge, no study has been been done on leveraging recurrent gated architectures for video segmentation. Accordingly, we propose a novel method for online segmentation of video sequences that incorporates temporal data. The network is built from fully convolutional element and recurrent unit that works on a sliding window over the temporal data. We also introduce a novel convolutional gated recurrent unit that preserves the spatial information and reduces the parameters learned. Our method has the advantage that it can work in an online fashion instead of operating over the whole input batch of video frames. The network is tested on the change detection dataset, and proved to have 5.5% improvement in F-measure over a plain fully convolutional network for per frame segmentation. It was also shown to have improvement of 1.4% for the F-measure compared to our baseline network that we call FCN 12s.