mlsanityData to 7 Oct 2026

Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning

arXiv 2210.08823 ↗First posted 17 Oct 2022cs.CVcited by 292

Not on the map. Why?

Cited by 20 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Existing fine-tuning methods either tune all parameters of the pre-trained model (full fine-tuning), which is not efficient, or only tune the last linear layer (linear probing), which suffers a significant accuracy drop compared to the full fine-tuning. In this paper, we propose a new parameter-efficient fine-tuning method termed as SSF, representing that researchers only need to Scale and Shift the deep Features extracted by a pre-trained model to catch up with the performance of full fine-tuning. In this way, SSF also surprisingly outperforms other parameter-efficient fine-tuning approaches even with a smaller number of tunable parameters. Furthermore, different from some existing parameter-efficient fine-tuning methods (e.g., Adapter or VPT) that introduce the extra parameters and computational cost in the training and inference stages, SSF only adds learnable parameters during the training stage, and these additional parameters can be merged into the original pre-trained model weights via re-parameterization in the inference phase. With the proposed SSF, our model obtains 2.46% (90.72% vs. 88.54%) and 11.48% (73.10% vs. 65.57%) performance improvement on FGVC and VTAB-1k in terms of Top-1 accuracy compared to the full fine-tuning but only fine-tuning about 0.3M parameters. We also conduct amounts of experiments in various model families (CNNs, Transformers, and MLPs) and datasets. Results on 26 image classification datasets in total and 3 robustness & out-of-distribution datasets show the effectiveness of SSF. Code is available at https://github.com/dongzelian/SSF.

Key facts. Cited by 292 papers in the map (data through 7 Oct 2026). Builds on 74 of its 112 references that are in the map. First posted on arXiv 17 Oct 2022.

Builds on

74 of 112 references.

References · 74

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2017Mask R-CNNcited by 7,259
38 more
  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2016Layer Normalizationcited by 5,579
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  11. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  12. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  13. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  14. 2013Describing Textures in the Wildcited by 2,110
  15. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  16. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  17. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  18. 2022Visual Prompt Tuningcited by 1,674
  19. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  20. 2018Group Normalizationcited by 1,514
  21. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  22. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  23. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  24. 2015Spatial Transformer Networkscited by 1,423
  25. 2019Natural Adversarial Examplescited by 1,401
  26. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  27. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  28. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  29. 2021Video Swin Transformercited by 1,090
  30. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  31. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  32. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  33. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  34. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  35. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  36. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  37. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  38. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 71; 2024: 104; 2025: 68; 2026: 43.

292 papers.

Sort:

By area:

  1. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  2. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  3. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
  4. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  5. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domaincited by 181
  6. 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 164
  7. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 162
  8. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  9. 2022Task Residual for Tuning Vision-Language Modelscited by 122
  10. 2023Parameter-Efficient Orthogonal Finetuning via Butterfly Factorizationcited by 112
  11. 2024Low-Rank Few-Shot Adaptation of Vision-Language Modelscited by 103
  12. 2024Model Stock: All we need is just a few fine-tuned modelscited by 87
38 more
  1. 2023One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuningcited by 84
  2. 2023DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuningcited by 82
  3. 2023VPGTrans: Transfer Visual Prompt Generator across LLMscited by 79
  4. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  5. 2023TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integrationcited by 76
  6. 2023Sensitivity-Aware Visual Parameter-Efficient Fine-Tuningcited by 74
  7. 2024Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactivelycited by 74
  8. 2023Towards Efficient Visual Adaption via Structural Re-parameterizationcited by 73
  9. 2023GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graphcited by 69
  10. 2024Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything Modelcited by 67
  11. 2024Task-Customized Mixture of Adapters for General Image Fusioncited by 59
  12. 2023Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurtscited by 58
  13. 2022Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learningcited by 57
  14. 2023Diversity-Aware Meta Visual Promptingcited by 57
  15. 2023Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunitycited by 54
  16. 2024Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysiscited by 52
  17. 2023Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancycited by 44
  18. 2024Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologiescited by 44
  19. 2024Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learningcited by 43
  20. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43
  21. 2023Gradient-based Parameter Selection for Efficient Fine-Tuningcited by 40
  22. 2024Advancing Parameter Efficiency in Fine-tuning via Representation Editingcited by 37
  23. 2023Efficient Adaptation of Large Vision Transformer via Adapter Re-Composingcited by 35
  24. 2024MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learningcited by 35
  25. 2023Visual Tuningcited by 34
  26. 20245%>100%: Breaking Performance Shackles of Full Fine-Tuning on Visual Recognition Taskscited by 34
  27. 2024Revisiting the Power of Prompt for Visual Tuningcited by 32
  28. 2023Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learningcited by 31
  29. 2024Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Modelscited by 31
  30. 2022iQuery: Instruments as Queries for Audio-Visual Sound Separationcited by 29
  31. 2024Time-, Memory- and Parameter-Efficient Visual Adaptationcited by 28
  32. 2023Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptationcited by 27
  33. 2024MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detectioncited by 27
  34. 2024Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectivescited by 27
  35. 2023UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memorycited by 26
  36. 2023Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbonecited by 26
  37. 2024Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptationcited by 26
  38. 2023G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networkscited by 25

50 most cited of 292 All 292 (JSON)