# MUNet: Motion Uncertainty-aware Semi-supervised Video Object Segmentation

arXiv: https://arxiv.org/abs/2111.14646 · HTML: https://mlsanity.com/paper/2111.14646/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 54 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [One-Shot Video Object Segmentation](https://mlsanity.com/paper/1611.05198/) (2016)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [The Secrets of Salient Object Segmentation](https://mlsanity.com/paper/1406.2807/) (2014)
- [FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation](https://mlsanity.com/paper/1902.09513/) (2019)
- [Learning Video Object Segmentation from Static Images](https://mlsanity.com/paper/1612.02646/) (2016)
- [Collaborative Video Object Segmentation by Foreground-Background Integration](https://mlsanity.com/paper/2003.08333/) (2020)
- [Video Object Segmentation with Episodic Graph Memory Networks](https://mlsanity.com/paper/2007.07020/) (2020)
- [Kernelized Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2007.08270/) (2020)
- [Online Adaptation of Convolutional Neural Networks for Video Object Segmentation](https://mlsanity.com/paper/1706.09364/) (2017)
- [Video Object Segmentation with Adaptive Feature Bank and Uncertain-Region Refinement](https://mlsanity.com/paper/2010.07958/) (2020)
- [Efficient Video Object Segmentation via Network Modulation](https://mlsanity.com/paper/1802.01218/) (2018)
- [Video Object Segmentation Without Temporal Information](https://mlsanity.com/paper/1709.06031/) (2017)
- [SegFlow: Joint Learning for Video Object Segmentation and Optical Flow](https://mlsanity.com/paper/1709.06750/) (2017)
- [Learning What to Learn for Video Object Segmentation](https://mlsanity.com/paper/2003.11540/) (2020)
- [Efficient Regional Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2103.12934/) (2021)
- [PReMVOS: Proposal-generation, Refinement and Merging for Video Object Segmentation](https://mlsanity.com/paper/1807.09190/) (2018)
- [EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow](https://mlsanity.com/paper/1501.02565/) (2015)
- [Learning Fast and Robust Target Models for Video Object Segmentation](https://mlsanity.com/paper/2003.00908/) (2020)
- [Motion Guided Attention for Video Salient Object Detection](https://mlsanity.com/paper/1909.07061/) (2019)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [A Generative Appearance Model for End-to-end Video Object Segmentation](https://mlsanity.com/paper/1811.11611/) (2018)
- [Fast Video Object Segmentation using the Global Context Module](https://mlsanity.com/paper/2001.11243/) (2020)
- [Learning Position and Target Consistency for Memory-based Video Object Segmentation](https://mlsanity.com/paper/2104.04329/) (2021)

## Built on by (most cited first, 7 of 7)

- [TAVGBench: Benchmarking Text to Audible-Video Generation](https://mlsanity.com/paper/2404.14381/) (2024)
- [Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation](https://mlsanity.com/paper/2207.02201/) (2022)
- [Fine-grained Audible Video Description](https://mlsanity.com/paper/2303.15616/) (2023)
- [Linear Video Transformer with Feature Fixation](https://mlsanity.com/paper/2210.08164/) (2022)
- [Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2309.11707/) (2023)
- [Few-Shot Learning in Video and 3D Object Detection: A Survey](https://mlsanity.com/paper/2507.17079/) (2025)
- [A General Approach to Visualizing Uncertainty in Statistical Graphics](https://mlsanity.com/paper/2508.00937/) (2025)

## Abstract (arXiv)

The task of semi-supervised video object segmentation (VOS) has been greatly advanced and state-of-the-art performance has been made by dense matching-based methods. The recent methods leverage space-time memory (STM) networks and learn to retrieve relevant information from all available sources, where the past frames with object masks form an external memory and the current frame as the query is segmented using the mask information in the memory. However, when forming the memory and performing matching, these methods only exploit the appearance information while ignoring the motion information. In this paper, we advocate the return of the motion information and propose a motion uncertainty-aware framework (MUNet) for semi-supervised VOS. First, we propose an implicit method to learn the spatial correspondences between neighboring frames, building upon a correlation cost volume. To handle the challenging cases of occlusion and textureless regions during constructing dense correspondences, we incorporate the uncertainty in dense matching and achieve motion uncertainty-aware feature representation. Second, we introduce a motion-aware spatial attention module to effectively fuse the motion feature with the semantic feature. Comprehensive experiments on challenging benchmarks show that textbftextitusing a small amount of data and combining it with powerful motion information can bring a significant performance boost. We achieve 76.5% J & F only using DAVIS17 for training, which significantly outperforms the textitSOTA methods under the low-data protocol. textitThe code will be released.