Revealing the Semantic Selection Gap in DINOv3 through Training-Free Few-Shot Segmentation
Not on the map. Why?
Abstract (arXiv)
Recent self-supervised Vision Transformers (ViTs), such as DINOv3, provide rich feature representations for dense vision tasks. This study investigates the intrinsic few-shot semantic segmentation (FSS) capabilities of frozen DINOv3 features through a training-free baseline, FSSDINO, utilizing class-specific prototypes and Gram-matrix refinement. Our results across binary, multi-class, and cross-domain (CDFSS) benchmarks demonstrate that this minimal approach, applied to the final backbone layer, is highly competitive with specialized methods involving complex decoders or test-time adaptation. Crucially, we conduct an Oracle-guided layer analysis, identifying a significant performance gap between the standard last-layer features and globally optimal intermediate representations. We reveal a "Safest vs. Optimal" dilemma: while the Oracle proves higher performance is attainable, matching the results of compute-intensive adaptation methods, current unsupervised and support-guided selection metrics consistently yield lower performance than the last-layer baseline. This characterizes a "Semantic Selection Gap" in Foundation Models, a disconnect where traditional heuristics fail to reliably identify high-fidelity features. Our work establishes the "Last-Layer" as a deceptively strong baseline and provides a rigorous diagnostic of the latent semantic potentials in DINOv3.The code is publicly available at https://github.com/hussni0997/fssdino.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 23 of its 26 references that are in the map. First posted on arXiv 7 Feb 2026.
Builds on
23 of 26 references.
References · 23
- 2023Segment Anythingcited by 8,158
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 657
- 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
- 2017One-Shot Learning for Semantic Segmentationcited by 337
- 2018DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Imagescited by 276
- 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
- 2023Personalize Segment Anything Model with One Shotcited by 250
- 2021Adaptive Prototype Learning and Allocation for Few-Shot Segmentationcited by 207
- 2023Matcher: Segment Anything with One Shot Using All-Purpose Feature Matchingcited by 142
- 2020Semantic Segmentation of Underwater Imagery: Dataset and Benchmarkcited by 95
- 2024VRP-SAM: SAM with Visual Reference Promptcited by 75
11 more
- 2022Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentationcited by 65
- 2022Feature-Proxy Transformer for Few-Shot Segmentationcited by 58
- 2022Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentationcited by 37
- 2024Bridge the Points: Graph-based Few-shot Segment Anything Semanticallycited by 35
- 2024Cross-Domain Few-Shot Segmentation via Iterative Support-Query Correspondence Miningcited by 30
- 2024Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentationcited by 22
- 2021MFNet: Multi-class Few-shot Segmentation Network with Pixel-wise Metric Learningcited by 10
- 2024Label Anything: Multi-Class Few-Shot Semantic Segmentation with Visual Promptscited by 9
- 2024Adaptive Layer Selection for Efficient Vision Transformer Fine-Tuningcited by 9
- 2024SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-promptingcited by 9
- 2025ViT-Split: Unleashing the Power of Vision Foundation Models via Efficient Splitting Headscited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.