mlsanityData to 7 Oct 2026

Learning Robust Correlation with Foundation Model for Weakly-Supervised Few-Shot Segmentation

arXiv 2405.19638 ↗First posted 30 May 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing few-shot segmentation (FSS) only considers learning support-query correlation and segmenting unseen categories under the precise pixel masks. However, the cost of a large number of pixel masks during training is expensive. This paper considers a more challenging scenario, weakly-supervised few-shot segmentation (WS-FSS), which only provides category (i.e. image-level) labels. It requires the model to learn robust support-query information when the generated mask is inaccurate. In this work, we design a Correlation Enhancement Network (CORENet) with foundation model, which utilizes multi-information guidance to learn robust correlation. Specifically, correlation-guided transformer (CGT) utilizes self-supervised ViT tokens to learn robust correlation from both local and global perspectives. From the perspective of semantic categories, the class-guided module (CGM) guides the model to locate valuable correlations through the pre-trained CLIP. Finally, the embedding-guided module (EGM) implicitly guides the model to supplement the inevitable information loss during the correlation learning by the original appearance embedding and finally generates the query mask. Extensive experiments on PASCAL-5i and COCO-20i have shown that CORENet exhibits excellent performance compared to existing methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 60 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

44 of 60 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  10. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  11. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  12. 2016Matching Networks for One Shot Learningcited by 2,246
32 more
  1. 2018Group Normalizationcited by 1,514
  2. 2021Scaling Vision Transformerscited by 993
  3. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  4. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  5. 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
  6. 2017One-Shot Learning for Semantic Segmentationcited by 337
  7. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  8. 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
  9. 2019CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learningcited by 217
  10. 2021Adaptive Prototype Learning and Allocation for Few-Shot Segmentationcited by 207
  11. 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
  12. 2019Feature Weighting and Boosting for Few-Shot Segmentationcited by 189
  13. 2015Weakly Supervised Deep Detection Networkscited by 183
  14. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  15. 2020Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processingcited by 161
  16. 2014Simultaneous Detection and Segmentationcited by 158
  17. 2022Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformerscited by 118
  18. 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
  19. 2020CRNet: Cross-Reference Networks for Few-Shot Segmentationcited by 101
  20. 2023Hierarchical Dense Correlation Distillation for Few-Shot Segmentationcited by 84
  21. 2021Mining Latent Classes for Few-shot Segmentationcited by 69
  22. 2022Feature-Proxy Transformer for Few-Shot Segmentationcited by 58
  23. 2023Learning Mask-aware CLIP Representations for Zero-Shot Segmentationcited by 57
  24. 2022Integrative Few-Shot Learning for Classification and Segmentationcited by 25
  25. 2022Interclass Prototype Relation for Few-Shot Segmentationcited by 25
  26. 2023MIANet: Aggregating Unbiased Instance and General Information for Few-Shot Semantic Segmentationcited by 25
  27. 2023Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentationcited by 24
  28. 2021A Pixel-Level Meta-Learner for Weakly Supervised Few-Shot Semantic Segmentationcited by 10
  29. 2022Improving Few-Shot Part Segmentation using Coarse Supervisioncited by 10
  30. 2023Iterative Few-shot Semantic Segmentation from Image Label Textcited by 10
  31. 2021Weakly Supervised Few-Shot Segmentation Via Meta-Learningcited by 9
  32. 2018Prior-Knowledge and Attention-based Meta-Learning for Few-Shot Learningcited by 1

Built on by

Not cited yet.