# Recurrent Instance Segmentation

arXiv: https://arxiv.org/abs/1511.08250 · HTML: https://mlsanity.com/paper/1511.08250/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2015.

## Built on by (most cited first, 21 of 21)

- [Pix2seq: A Language Modeling Framework for Object Detection](https://mlsanity.com/paper/2109.10852/) (2021)
- [ISTR: End-to-End Instance Segmentation with Transformers](https://mlsanity.com/paper/2105.00637/) (2021)
- [Object Detection with Transformers: A Review](https://mlsanity.com/paper/2306.04670/) (2023)
- [Learning Equivariant Segmentation with Instance-Unique Querying](https://mlsanity.com/paper/2210.00911/) (2022)
- [Hierarchical Approach for Joint Semantic, Plant Instance, and Leaf Instance Segmentation in the Agricultural Domain](https://mlsanity.com/paper/2210.07879/) (2022)
- [Unknown Object Segmentation from Stereo Images](https://mlsanity.com/paper/2103.06796/) (2021)
- [GrowliFlower: An image time series dataset for GROWth analysis of cauLIFLOWER](https://mlsanity.com/paper/2204.00294/) (2022)
- [A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture](https://mlsanity.com/paper/2210.11318/) (2022)
- [ISDA: Position-Aware Instance Segmentation with Deformable Attention](https://mlsanity.com/paper/2202.12251/) (2022)
- [Combining Deep Learning and Mathematical Morphology for Historical Map Segmentation](https://mlsanity.com/paper/2101.02144/) (2021)
- [Object-Guided Instance Segmentation With Auxiliary Feature Refinement for Biological Images](https://mlsanity.com/paper/2106.07159/) (2021)
- [Segment, Select, Correct: A Framework for Weakly-Supervised Referring Segmentation](https://mlsanity.com/paper/2310.13479/) (2023)
- [Zero-shot Hierarchical Plant Segmentation via Foundation Segmentation Models and Text-to-image Attention](https://mlsanity.com/paper/2509.09116/) (2025)
- [H-EMD: A Hierarchical Earth Mover's Distance Method for Instance Segmentation](https://mlsanity.com/paper/2206.01309/) (2022)
- [Analysis of the performance of U-Net neural networks for the segmentation of living cells](https://mlsanity.com/paper/2210.01538/) (2022)
- [Non-parametric Memory for Spatio-Temporal Segmentation of Construction Zones for Self-Driving](https://mlsanity.com/paper/2101.06865/) (2021)
- [Multi-Class Multi-Instance Count Conditioned Adversarial Image Generation](https://mlsanity.com/paper/2103.16795/) (2021)
- [LeafMask: Towards Greater Accuracy on Leaf Segmentation](https://mlsanity.com/paper/2108.03568/) (2021)
- [COVID-19 Lung Lesion Segmentation Using a Sparsely Supervised Mask R-CNN on Chest X-rays Automatically Computed from Volumetric CTs](https://mlsanity.com/paper/2105.08147/) (2021)
- [Localizing the Recurrent Laryngeal Nerve via Ultrasound with a Bayesian Shape Framework](https://mlsanity.com/paper/2206.15254/) (2022)
- [Deformable Image Registration using Unsupervised Deep Learning for CBCT-guided Abdominal Radiotherapy](https://mlsanity.com/paper/2208.13686/) (2022)

## Abstract (arXiv)

Instance segmentation is the problem of detecting and delineating each distinct object of interest appearing in an image. Current instance segmentation approaches consist of ensembles of modules that are trained independently of each other, thus missing opportunities for joint learning. Here we propose a new instance segmentation paradigm consisting in an end-to-end method that learns how to segment instances sequentially. The model is based on a recurrent neural network that sequentially finds objects and their segmentations one at a time. This net is provided with a spatial memory that keeps track of what pixels have been explained and allows occlusion handling. In order to train the model we designed a principled loss function that accurately represents the properties of the instance segmentation problem. In the experiments carried out, we found that our method outperforms recent approaches on multiple person segmentation, and all state of the art approaches on the Plant Phenotyping dataset for leaf counting.