mlsanityData to 7 Oct 2026

Applying ViT in Generalized Few-shot Semantic Segmentation

arXiv 2408.14957 ↗First posted 27 Aug 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper explores the capability of ViT-based models under the generalized few-shot semantic segmentation (GFSS) framework. We conduct experiments with various combinations of backbone models, including ResNets and pretrained Vision Transformer (ViT)-based models, along with decoders featuring a linear classifier, UPerNet, and Mask Transformer. The structure made of DINOv2 and linear classifier takes the lead on popular few-shot segmentation bench mark PASCAL-5i, substantially outperforming the best of ResNet structure by 116% in one-shot scenario. We demonstrate the great potential of large pretrained ViT-based model on GFSS task, and expect further improvement on testing benchmarks. However, a potential caveat is that when applying pure ViT-based model and large scale ViT decoder, the model is easy to overfit.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 27 Aug 2024.

Builds on

20 of 27 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  7. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  8. 2016Pyramid Scene Parsing Networkcited by 2,472
  9. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  10. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  12. 2020TransTrack: Multiple Object Tracking with Transformercited by 370
8 more
  1. 2021Adaptive Prototype Learning and Allocation for Few-Shot Segmentationcited by 207
  2. 2020Prototype Mixture Models for Few-shot Semantic Segmentationcited by 190
  3. 2020Few-Shot Segmentation Without Meta-Learning: A Good Transductive Inference Is All You Need?cited by 135
  4. 2022Learning What Not to Segment: A New Perspective on Few-Shot Segmentationcited by 120
  5. 2021Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformercited by 116
  6. 2018Memory Matching Networks for One-Shot Image Recognitioncited by 65
  7. 2022Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentationcited by 37
  8. 2022A Strong Baseline for Generalized Few-Shot Semantic Segmentationcited by 22

Built on by

Not cited yet.