# Spatiotemporal Graph Neural Network based Mask Reconstruction for Video Object Segmentation

arXiv: https://arxiv.org/abs/2012.05499 · HTML: https://mlsanity.com/paper/2012.05499/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2020.

## Built on by (most cited first, 12 of 12)

- [Hierarchical Memory Matching Network for Video Object Segmentation](https://mlsanity.com/paper/2109.11404/) (2021)
- [You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos](https://mlsanity.com/paper/2303.07863/) (2023)
- [Tackling Background Distraction in Video Object Segmentation](https://mlsanity.com/paper/2207.06953/) (2022)
- [Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Network](https://mlsanity.com/paper/2412.15678/) (2024)
- [Declaration-based Prompt Tuning for Visual Question Answering](https://mlsanity.com/paper/2205.02456/) (2022)
- [Jointly Visual- and Semantic-Aware Graph Memory Networks for Temporal Sentence Localization in Videos](https://mlsanity.com/paper/2303.01046/) (2023)
- [FAMINet: Learning Real-time Semi-supervised Video Object Segmentation with Steepest Optimized Optical Flow](https://mlsanity.com/paper/2111.10531/) (2021)
- [Transform-Equivariant Consistency Learning for Temporal Sentence Grounding](https://mlsanity.com/paper/2305.04123/) (2023)
- [Joint Top-Down and Bottom-Up Frameworks for 3D Visual Grounding](https://mlsanity.com/paper/2410.15615/) (2024)
- [Pixel-Level Equalized Matching for Video Object Segmentation](https://mlsanity.com/paper/2209.03139/) (2022)
- [Hard-Label Black-Box Attacks on 3D Point Clouds](https://mlsanity.com/paper/2412.00404/) (2024)
- [First-place Solution for Streetscape Shop Sign Recognition Competition](https://mlsanity.com/paper/2501.02811/) (2025)

## Abstract (arXiv)

This paper addresses the task of segmenting class-agnostic objects in semi-supervised setting. Although previous detection based methods achieve relatively good performance, these approaches extract the best proposal by a greedy strategy, which may lose the local patch details outside the chosen candidate. In this paper, we propose a novel spatiotemporal graph neural network (STG-Net) to reconstruct more accurate masks for video object segmentation, which captures the local contexts by utilizing all proposals. In the spatial graph, we treat object proposals of a frame as nodes and represent their correlations with an edge weight strategy for mask context aggregation. To capture temporal information from previous frames, we use a memory network to refine the mask of current frame by retrieving historic masks in a temporal graph. The joint use of both local patch details and temporal relationships allow us to better address the challenges such as object occlusion and missing. Without online learning and fine-tuning, our STG-Net achieves state-of-the-art performance on four large benchmarks (DAVIS, YouTube-VOS, SegTrack-v2, and YouTube-Objects), demonstrating the effectiveness of the proposed approach.