mlsanityData to 7 Oct 2026

Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning

arXiv 2206.06122 ↗First posted 13 Jun 2022cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

Freezing the pre-trained backbone has become a standard paradigm to avoid overfitting in few-shot segmentation. In this paper, we rethink the paradigm and explore a new regime: em fine-tuning a small part of parameters in the backbone. We present a solution to overcome the overfitting problem, leading to better model generalization on learning novel classes. Our method decomposes backbone parameters into three successive matrices via the Singular Value Decomposition (SVD), then em only fine-tunes the singular values and keeps others frozen. The above design allows the model to adjust feature representations on novel classes while maintaining semantic clues within the pre-trained backbone. We evaluate our em Singular Value Fine-tuning (SVF) approach on various few-shot segmentation methods with different backbones. We achieve state-of-the-art results on both Pascal-5i and COCO-20i across 1-shot and 5-shot settings. Hopefully, this simple baseline will encourage researchers to rethink the role of backbone fine-tuning in few-shot settings. The source code and models will be available at https://github.com/syp2ysy/SVF.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 45 of its 57 references that are in the map. First posted on arXiv 13 Jun 2022.

Builds on

45 of 57 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2017Mask R-CNNcited by 7,259
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  11. 2015Fast R-CNNcited by 2,930
  12. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
33 more
  1. 2022Visual Prompt Tuningcited by 1,674
  2. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  3. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  4. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
  5. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  6. 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
  7. 2017One-Shot Learning for Semantic Segmentationcited by 337
  8. 2020K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapterscited by 318
  9. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  10. 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
  11. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  12. 2020Revisiting Few-sample BERT Fine-tuningcited by 237
  13. 2019CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learningcited by 217
  14. 2021Adaptive Prototype Learning and Allocation for Few-Shot Segmentationcited by 207
  15. 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
  16. 2020Prototype Mixture Models for Few-shot Semantic Segmentationcited by 190
  17. 2019Feature Weighting and Boosting for Few-Shot Segmentationcited by 189
  18. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  19. 2018SG-One: Similarity Guidance Network for One-Shot Semantic Segmentationcited by 169
  20. 2018SpotTune: Transfer Learning through Adaptive Fine-tuningcited by 166
  21. 2020Part-aware Prototype Network for Few-shot Semantic Segmentationcited by 162
  22. 2021Few-Shot Segmentation via Cycle-Consistent Transformercited by 133
  23. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  24. 2022Learning What Not to Segment: A New Perspective on Few-Shot Segmentationcited by 120
  25. 2021Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformercited by 116
  26. 2018The Singular Values of Convolutional Layerscited by 95
  27. 2020Feature Pyramid Transformercited by 94
  28. 2021Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuningcited by 82
  29. 2021Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learningcited by 72
  30. 2021Mining Latent Classes for Few-shot Segmentationcited by 69
  31. 2021CTNet: Context-based Tandem Network for Semantic Segmentationcited by 43
  32. 2016Improving training of deep neural networks via Singular Value Boundingcited by 26
  33. 2021Few-Shot Segmentation with Global and Local Contrastive Learningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 14; 2024: 15; 2025: 11; 2026: 7.

50 papers.

Sort:

By area:

  1. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  2. 2024VRP-SAM: SAM with Visual Reference Promptcited by 75
  3. 2023Learning Mask-aware CLIP Representations for Zero-Shot Segmentationcited by 57
  4. 2024SVFT: Parameter-Efficient Fine-Tuning with Singular Vectorscited by 57
  5. 2023Exploring Effective Factors for Improving Visual In-Context Learningcited by 43
  6. 2024Bridge the Points: Graph-based Few-shot Segment Anything Semanticallycited by 35
  7. 2023M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognitioncited by 34
  8. 2023Self-Calibrated Cross Attention Network for Few-Shot Segmentationcited by 32
  9. 2023SAM-PARSER: Fine-tuning SAM Efficiently by Parameter Space Reconstructioncited by 23
  10. 2022A Strong Baseline for Generalized Few-Shot Semantic Segmentationcited by 22
  11. 2024Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentationcited by 22
  12. 2024Hybrid Mamba for Few-Shot Segmentationcited by 21
38 more
  1. 2024Eliminating Feature Ambiguity for Few-Shot Segmentationcited by 19
  2. 2022Prediction Calibration for Generalized Few-shot Semantic Segmentationcited by 16
  3. 2025Visual Position Prompt for MLLM based Visual Groundingcited by 16
  4. 2024A Novel Benchmark for Few-Shot Semantic Segmentation in the Era of Foundation Modelscited by 13
  5. 2024Explore In-Context Segmentation via Latent Diffusion Modelscited by 12
  6. 2023Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensingcited by 11
  7. 2024KIND: Knowledge Integration and Diversion for Training Decomposable Modelscited by 10
  8. 2024Prompt-and-Transfer: Dynamic Class-aware Enhancement for Few-shot Segmentationcited by 10
  9. 2025DSV-LFS: Unifying LLM-Driven Semantic Cues with Visual Features for Robust Few-Shot Segmentationcited by 6
  10. 2025ESSA: Evolutionary Strategies for Scalable Alignmentcited by 6
  11. 2026Model Merging in the Essential Subspacecited by 6
  12. 2023Dense Affinity Matching for Few-Shot Segmentationcited by 5
  13. 2023PRSeg: A Lightweight Patch Rotate MLP Decoder for Semantic Segmentationcited by 4
  14. 2025Accelerating Volumetric Medical Image Annotation via Short-Long Memory SAM 2cited by 4
  15. 2025MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptationcited by 4
  16. 2023Decomposition Ascribed Synergistic Learning for Unified Image Restorationcited by 3
  17. 2024Towards Real Zero-Shot Camouflaged Object Segmentation without Camouflaged Annotationscited by 3
  18. 2025CLIP-SVD: Efficient and Interpretable Vision-Language Adaptation via Singular Valuescited by 3
  19. 2022Domain-invariant Prototypes for Semantic Segmentationcited by 2
  20. 2023ENInst: Enhancing Weakly-supervised Low-shot Instance Segmentationcited by 2
  21. 2023SVD-DIP: Overcoming the Overfitting Problem in DIP-based CT Reconstructioncited by 2
  22. 2024MetaSeg: Content-Aware Meta-Net for Omni-Supervised Semantic Segmentationcited by 2
  23. 2024Show and Grasp: Few-shot Semantic Segmentation for Robot Grasping through Zero-shot Foundation Modelscited by 2
  24. 2023TVE: Learning Meta-attribution for Transferable Vision Explainercited by 1
  25. 2024Inference-Time Rule Eraser: Fair Recognition via Distilling and Removing Biased Rulescited by 1
  26. 2025MITS: A Large-Scale Multimodal Benchmark Dataset for Intelligent Traffic Surveillancecited by 1
  27. 2025FS-SAM2: Adapting Segment Anything Model 2 for Few-Shot Semantic Segmentation via Low-Rank Adaptationcited by 1
  28. 2025Take a Peek: Efficient Encoder Adaptation for Few-Shot Semantic Segmentation via LoRAcited by 1
  29. 2026Quantifying the Uncertainty of Foundation Models with Singular Value Ensemblescited by 1
  30. 2026Essential Subspace Merging for Multi-Task Learningcited by 1
  31. 2023Background Clustering Pre-training for Few-shot Segmentationcited by 0
  32. 2024Image to Pseudo-Episode: Boosting Few-Shot Segmentation by Unlabeled Datacited by 0
  33. 2025FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Promptscited by 0
  34. 2025FRoD: Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergencecited by 0
  35. 2026Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuningcited by 0
  36. 2026SLAD : Shared LoRA Adapters for Task Specific Distillationcited by 0
  37. 2026P3-SAM: SAM with Perceptual Parallel Prompt for Few-Shot Strip Steel Surface Defect Segmentationcited by 0
  38. 2026Reuse or Relearn? A Spectral View of Earth Observation Foundation Modelscited by 0