Holistic Prototype Attention Network for Few-Shot VOS
Not on the map. Why?
Abstract (arXiv)
Few-shot video object segmentation (FSVOS) aims to segment dynamic objects of unseen classes by resorting to a small set of support images that contain pixel-level object annotations. Existing methods have demonstrated that the domain agent-based attention mechanism is effective in FSVOS by learning the correlation between support images and query frames. However, the agent frame contains redundant pixel information and background noise, resulting in inferior segmentation performance. Moreover, existing methods tend to ignore inter-frame correlations in query videos. To alleviate the above dilemma, we propose a holistic prototype attention network (HPAN) for advancing FSVOS. Specifically, HPAN introduces a prototype graph attention module (PGAM) and a bidirectional prototype attention module (BPAM), transferring informative knowledge from seen to unseen classes. PGAM generates local prototypes from all foreground features and then utilizes their internal correlations to enhance the representation of the holistic prototypes. BPAM exploits the holistic information from support images and video frames by fusing co-attention and self-attention to achieve support-query semantic consistency and inner-frame temporal consistency. Extensive experiments on YouTube-FSVOS have been provided to demonstrate the effectiveness and superiority of our proposed HPAN method.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 51 of its 72 references that are in the map. First posted on arXiv 16 Jul 2023.
Builds on
51 of 72 references.
References · 51
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
- 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
- 2017One-Shot Learning for Semantic Segmentationcited by 337
- 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
- 2016One-Shot Video Object Segmentationcited by 233
- 2021Associating Objects with Transformers for Video Object Segmentationcited by 225
- 2021Adaptive Prototype Learning and Allocation for Few-Shot Segmentationcited by 207
- 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 200
- 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
38 more
- 2020Prototype Mixture Models for Few-shot Semantic Segmentationcited by 190
- 2020Part-aware Prototype Network for Few-shot Semantic Segmentationcited by 162
- 2020Attribute Prototype Network for Zero-Shot Learningcited by 160
- 2022Rethinking Semantic Segmentation: A Prototype Viewcited by 140
- 2020Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need?cited by 135
- 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
- 2018Robust Classification with Convolutional Prototype Learningcited by 122
- 2022Learning What Not to Segment: A New Perspective on Few-Shot Segmentationcited by 120
- 2020Video Object Segmentation with Episodic Graph Memory Networkscited by 118
- 2021Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformercited by 116
- 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
- 2020MAST: A Memory-Augmented Self-supervised Trackercited by 109
- 2019Prototype Rectification for Few-Shot Learningcited by 96
- 2021Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentationcited by 95
- 2021SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentationcited by 87
- 2020Zero-Shot Video Object Segmentation via Attentive Graph Neural Networkscited by 84
- 2021Hierarchical Memory Matching Network for Video Object Segmentationcited by 72
- 2022Physical Adversarial Attack meets Computer Vision: A Decade Surveycited by 66
- 2020Motion-Attentive Transition for Zero-Shot Video Object Segmentationcited by 62
- 2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
- 2022Visual Recognition with Deep Nearest Centroidscited by 59
- 2022Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patchescited by 57
- 2020Learning Video Object Segmentation from Unlabeled Videoscited by 42
- 2022Per-Clip Video Object Segmentationcited by 40
- 2021TF-Blender: Temporal Feature Blender for Video Object Detectioncited by 37
- 2022Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentationcited by 37
- 2022Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentationcited by 37
- 2023CLUSTSEG: Clustering for Universal Segmentationcited by 32
- 2022SWEM: Towards Real-Time Video Object Segmentation with Sequential Weighted Expectation-Maximizationcited by 26
- 2022Global Spectral Filter Memory Network for Video Object Segmentationcited by 22
- 2023FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Networkcited by 22
- 2021Target-Aware Object Discovery and Association for Unsupervised Video Multi-Object Segmentationcited by 21
- 2021Dense Gaussian Processes for Few-Shot Segmentationcited by 17
- 2023Multi-Granularity Denoising and Bidirectional Alignment for Weakly Supervised Semantic Segmentationcited by 16
- 2019Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot Learning on Category Graphcited by 15
- 2023Co-attention Propagation Network for Zero-Shot Video Object Segmentationcited by 9
- 2021Few-Shot Video Object Detectioncited by 7
- 2022Temporal Transductive Inference for Few-Shot Video Object Segmentationcited by 6
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2024: 1.
1 paper.