# CompFeat: Comprehensive Feature Aggregation for Video Instance Segmentation

arXiv: https://arxiv.org/abs/2012.03400 · HTML: https://mlsanity.com/paper/2012.03400/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2020.

## Built on by (most cited first, 30 of 53)

- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [Video Instance Segmentation using Inter-Frame Communication Transformers](https://mlsanity.com/paper/2106.03299/) (2021)
- [SeqFormer: Sequential Transformer for Video Instance Segmentation](https://mlsanity.com/paper/2112.08275/) (2021)
- [In Defense of Online Models for Video Instance Segmentation](https://mlsanity.com/paper/2207.10661/) (2022)
- [VITA: Video Instance Segmentation via Object Token Association](https://mlsanity.com/paper/2206.04403/) (2022)
- [Crossover Learning for Fast Online Video Instance Segmentation](https://mlsanity.com/paper/2104.05970/) (2021)
- [Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation](https://mlsanity.com/paper/2204.04656/) (2022)
- [Temporally Efficient Vision Transformer for Video Instance Segmentation](https://mlsanity.com/paper/2204.08412/) (2022)
- [A Generalized Framework for Video Instance Segmentation](https://mlsanity.com/paper/2211.08834/) (2022)
- [TubeFormer-DeepLab: Video Mask Transformer](https://mlsanity.com/paper/2205.15361/) (2022)
- [Efficient Video Instance Segmentation via Tracklet Query and Proposal](https://mlsanity.com/paper/2203.01853/) (2022)
- [TarViS: A Unified Approach for Target-based Video Segmentation](https://mlsanity.com/paper/2301.02657/) (2023)
- [VISOLO: Grid-Based Space-Time Aggregation for Efficient Online Video Instance Segmentation](https://mlsanity.com/paper/2112.04177/) (2021)
- [D^2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos](https://mlsanity.com/paper/2111.07774/) (2021)
- [Learning to Track Instances without Video Annotations](https://mlsanity.com/paper/2104.00287/) (2021)
- [TCOVIS: Temporally Consistent Online Video Instance Segmentation](https://mlsanity.com/paper/2309.11857/) (2023)
- [Improving Video Instance Segmentation via Temporal Pyramid Routing](https://mlsanity.com/paper/2107.13155/) (2021)
- [Hybrid Instance-aware Temporal Fusion for Online Video Instance Segmentation](https://mlsanity.com/paper/2112.01695/) (2021)
- [STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation](https://mlsanity.com/paper/2202.03747/) (2022)
- [InstanceFormer: An Online Video Instance Segmentation Framework](https://mlsanity.com/paper/2208.10547/) (2022)
- [InsPro: Propagating Instance Query and Proposal for Online Video Instance Segmentation](https://mlsanity.com/paper/2301.01882/) (2023)
- [CLIP-VIS: Adapting CLIP for Open-Vocabulary Video Instance Segmentation](https://mlsanity.com/paper/2403.12455/) (2024)
- [RankSeg: Adaptive Pixel Classification with Image Category Ranking for Segmentation](https://mlsanity.com/paper/2203.04187/) (2022)
- [Video Instance Segmentation via Multi-scale Spatio-temporal Split Attention Transformer](https://mlsanity.com/paper/2203.13253/) (2022)
- [Instance As Identity: A Generic Online Paradigm for Video Instance Segmentation](https://mlsanity.com/paper/2208.03079/) (2022)
- [NOVIS: A Case for End-to-End Near-Online Video Instance Segmentation](https://mlsanity.com/paper/2308.15266/) (2023)
- [Tracking Instances as Queries](https://mlsanity.com/paper/2106.11963/) (2021)
- [1st Place Solution for YouTubeVOS Challenge 2021:Video Instance Segmentation](https://mlsanity.com/paper/2106.06649/) (2021)

## Abstract (arXiv)

Video instance segmentation is a complex task in which we need to detect, segment, and track each object for any given video. Previous approaches only utilize single-frame features for the detection, segmentation, and tracking of objects and they suffer in the video scenario due to several distinct challenges such as motion blur and drastic appearance change. To eliminate ambiguities introduced by only using single-frame features, we propose a novel comprehensive feature aggregation approach (CompFeat) to refine features at both frame-level and object-level with temporal and spatial context information. The aggregation process is carefully designed with a new attention mechanism which significantly increases the discriminative power of the learned features. We further improve the tracking capability of our model through a siamese design by incorporating both feature similarities and spatial similarities. Experiments conducted on the YouTube-VIS dataset validate the effectiveness of proposed CompFeat. Our code will be available at https://github.com/SHI-Labs/CompFeat-for-Video-Instance-Segmentation.