# Learning Video Instance Segmentation with Recurrent Graph Neural Networks

arXiv: https://arxiv.org/abs/2012.03911 · HTML: https://mlsanity.com/paper/2012.03911/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2020.

## Built on by (most cited first, 6 of 6)

- [Crossover Learning for Fast Online Video Instance Segmentation](https://mlsanity.com/paper/2104.05970/) (2021)
- [Temporally Efficient Vision Transformer for Video Instance Segmentation](https://mlsanity.com/paper/2204.08412/) (2022)
- [Video Instance Segmentation via Multi-scale Spatio-temporal Split Attention Transformer](https://mlsanity.com/paper/2203.13253/) (2022)
- [Robust Online Video Instance Segmentation with Track Queries](https://mlsanity.com/paper/2211.09108/) (2022)
- [Fast Interactive Video Object Segmentation with Graph Neural Networks](https://mlsanity.com/paper/2103.03821/) (2021)
- [Deep Learning Techniques for Video Instance Segmentation: A Survey](https://mlsanity.com/paper/2310.12393/) (2023)

## Abstract (arXiv)

Most existing approaches to video instance segmentation comprise multiple modules that are heuristically combined to produce the final output. Formulating a purely learning-based method instead, which models both the temporal aspect as well as a generic track management required to solve the video instance segmentation task, is a highly challenging problem. In this work, we propose a novel learning formulation, where the entire video instance segmentation problem is modelled jointly. We fit a flexible model to our formulation that, with the help of a graph neural network, processes all available new information in each frame. Past information is considered and processed via a recurrent connection. We demonstrate the effectiveness of the proposed approach in comprehensive experiments. Our approach, operating at over 25 FPS, outperforms previous video real-time methods. We further conduct detailed ablative experiments that validate the different aspects of our approach.