# Target-Aware Object Discovery and Association for Unsupervised Video Multi-Object Segmentation

arXiv: https://arxiv.org/abs/2104.04782 · HTML: https://mlsanity.com/paper/2104.04782/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 71 references that are in the map.
- First posted on arXiv 10 Apr 2021.

## Builds on (most cited first, 30 of 52)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [CenterMask : Real-Time Anchor-Free Instance Segmentation](https://mlsanity.com/paper/1911.06667/) (2019)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation](https://mlsanity.com/paper/1902.09513/) (2019)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Deep Extreme Cut: From Extreme Points to Object Segmentation](https://mlsanity.com/paper/1711.09081/) (2017)
- [Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation](https://mlsanity.com/paper/1912.04573/) (2019)
- [STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos](https://mlsanity.com/paper/2003.08429/) (2020)
- [Semantic Instance Segmentation with a Discriminative Loss Function](https://mlsanity.com/paper/1708.02551/) (2017)
- [Video Object Segmentation with Episodic Graph Memory Networks](https://mlsanity.com/paper/2007.07020/) (2020)
- [Efficient Video Object Segmentation via Network Modulation](https://mlsanity.com/paper/1802.01218/) (2018)
- [The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation](https://mlsanity.com/paper/1905.00737/) (2019)
- [Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2103.14581/) (2021)

## Built on by (most cited first, 21 of 21)

- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.09653/) (2022)
- [Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook](https://mlsanity.com/paper/2112.02838/) (2021)
- [Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2207.08485/) (2022)
- [Locality-Aware Inter-and Intra-Video Reconstruction for Self-Supervised Correspondence Learning](https://mlsanity.com/paper/2203.14333/) (2022)
- [Boosting Object Representation Learning via Motion and Object Continuity](https://mlsanity.com/paper/2211.09771/) (2022)
- [UVOSAM: A Mask-free Paradigm for Unsupervised Video Object Segmentation via Segment Anything Model](https://mlsanity.com/paper/2305.12659/) (2023)
- [Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2309.11707/) (2023)
- [Out of Sight, Still in Mind: Reasoning and Planning about Unobserved Objects with Video Tracking Enabled Memory Models](https://mlsanity.com/paper/2309.15278/) (2023)
- [What is Point Supervision Worth in Video Instance Segmentation?](https://mlsanity.com/paper/2404.01990/) (2024)
- [UVIS: Unsupervised Video Instance Segmentation](https://mlsanity.com/paper/2406.06908/) (2024)
- [A Novel Long-term Iterative Mining Scheme for Video Salient Object Detection](https://mlsanity.com/paper/2206.09564/) (2022)
- [Surface-SOS: Self-Supervised Object Segmentation via Neural Surface Representation](https://mlsanity.com/paper/2501.09947/) (2025)
- [Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation](https://mlsanity.com/paper/2203.01217/) (2022)
- [Hierarchical Graph Pattern Understanding for Zero-Shot VOS](https://mlsanity.com/paper/2312.09525/) (2023)
- [MOD-UV: Learning Mobile Object Detectors from Unlabeled Videos](https://mlsanity.com/paper/2405.14841/) (2024)
- [Maximal Cliques on Multi-Frame Proposal Graph for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2301.12352/) (2023)
- [Holistic Prototype Attention Network for Few-Shot VOS](https://mlsanity.com/paper/2307.07933/) (2023)
- [GMOS: Grounding Moving Object Segmentation in 3D Space and Time](https://mlsanity.com/paper/2605.30352/) (2026)
- [Video Object of Interest Segmentation](https://mlsanity.com/paper/2212.02871/) (2022)
- [FTIO: Frequent Temporally Integrated Objects](https://mlsanity.com/paper/2508.16183/) (2025)

## Abstract (arXiv)

This paper addresses the task of unsupervised video multi-object segmentation. Current approaches follow a two-stage paradigm: 1) detect object proposals using pre-trained Mask R-CNN, and 2) conduct generic feature matching for temporal association using re-identification techniques. However, the generic features, widely used in both stages, are not reliable for characterizing unseen objects, leading to poor generalization. To address this, we introduce a novel approach for more accurate and efficient spatio-temporal segmentation. In particular, to address instance discrimination, we propose to combine foreground region estimation and instance grouping together in one network, and additionally introduce temporal guidance for segmenting each frame, enabling more accurate object discovery. For temporal association, we complement current video object segmentation architectures with a discriminative appearance model, capable of capturing more fine-grained target-specific information. Given object proposals from the instance discrimination network, three essential strategies are adopted to achieve accurate segmentation: 1) target-specific tracking using a memory-augmented appearance model; 2) target-agnostic verification to trace possible tracklets for the proposal; 3) adaptive memory updating using the verified segments. We evaluate the proposed approach on DAVIS17 and YouTube-VIS, and the results demonstrate that it outperforms state-of-the-art methods both in segmentation accuracy and inference speed.