# ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation

arXiv: https://arxiv.org/abs/1511.07053 · HTML: https://mlsanity.com/paper/1511.07053/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2015.

## Built on by (most cited first, 19 of 19)

- [Spatially-Adaptive Image Restoration using Distortion-Guided Networks](https://mlsanity.com/paper/2108.08617/) (2021)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [Sequencer: Deep LSTM for Image Classification](https://mlsanity.com/paper/2205.01972/) (2022)
- [Unsupervised Domain Adaptation for Semantic Image Segmentation: a Comprehensive Survey](https://mlsanity.com/paper/2112.03241/) (2021)
- [Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation](https://mlsanity.com/paper/2207.12964/) (2022)
- [Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review](https://mlsanity.com/paper/2203.10939/) (2022)
- [Cross-attention Spatio-temporal Context Transformer for Semantic Segmentation of Historical Maps](https://mlsanity.com/paper/2310.12616/) (2023)
- [Ultra-high Resolution Image Segmentation via Locality-aware Context Fusion and Alternating Local Enhancement](https://mlsanity.com/paper/2109.02580/) (2021)
- [Recurrent Feature Propagation and Edge Skip-Connections for Automatic Abdominal Organ Segmentation](https://mlsanity.com/paper/2201.00317/) (2022)
- [Combining unsupervised and supervised learning for predicting the final stroke lesion](https://mlsanity.com/paper/2101.00489/) (2021)
- [Quantifying Model Uncertainty for Semantic Segmentation using Operators in the RKHS](https://mlsanity.com/paper/2211.01999/) (2022)
- [Robust Unsupervised Domain Adaptation by Retaining Confident Entropy via Edge Concatenation](https://mlsanity.com/paper/2310.07149/) (2023)
- [Adversarially Domain-adaptive Latent Diffusion for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2412.16859/) (2024)
- [A Racing Dataset and Baseline Model for Track Detection in Autonomous Racing](https://mlsanity.com/paper/2502.14068/) (2025)
- [Do Neural Networks for Segmentation Understand Insideness?](https://mlsanity.com/paper/2201.10664/) (2022)
- [Progressively Dual Prior Guided Few-shot Semantic Segmentation](https://mlsanity.com/paper/2211.15467/) (2022)
- [Convolutional Networks as Extremely Small Foundation Models: Visual Prompting and Theoretical Perspective](https://mlsanity.com/paper/2409.10555/) (2024)
- [Real-Time Pedestrian Detection on IoT Edge Devices: A Lightweight Deep Learning Approach](https://mlsanity.com/paper/2409.15740/) (2024)
- [To GAN or Not To GAN: Segmentation Analysis on Mars DEM](https://mlsanity.com/paper/2606.13252/) (2026)

## Abstract (arXiv)

We propose a structured prediction architecture, which exploits the local generic features extracted by Convolutional Neural Networks and the capacity of Recurrent Neural Networks (RNN) to retrieve distant dependencies. The proposed architecture, called ReSeg, is based on the recently introduced ReNet model for image classification. We modify and extend it to perform the more challenging task of semantic segmentation. Each ReNet layer is composed of four RNN that sweep the image horizontally and vertically in both directions, encoding patches or activations, and providing relevant global information. Moreover, ReNet layers are stacked on top of pre-trained convolutional layers, benefiting from generic local features. Upsampling layers follow ReNet layers to recover the original image resolution in the final predictions. The proposed ReSeg architecture is efficient, flexible and suitable for a variety of semantic segmentation tasks. We evaluate ReSeg on several widely-used semantic segmentation datasets: Weizmann Horse, Oxford Flower, and CamVid; achieving state-of-the-art performance. Results show that ReSeg can act as a suitable architecture for semantic segmentation tasks, and may have further applications in other structured prediction problems. The source code and model hyperparameters are available on https://github.com/fvisin/reseg.