# Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning

arXiv: https://arxiv.org/abs/2102.03983 · HTML: https://mlsanity.com/paper/2102.03983/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 43 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [Large-Scale Evolution of Image Classifiers](https://mlsanity.com/paper/1703.01041/) (2017)
- [Few-Shot Learning with Graph Neural Networks](https://mlsanity.com/paper/1711.04043/) (2017)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Hierarchical Representations for Efficient Architecture Search](https://mlsanity.com/paper/1711.00436/) (2017)
- [Genetic CNN](https://mlsanity.com/paper/1703.01513/) (2017)
- [Few-Shot Image Recognition by Predicting Parameters from Activations](https://mlsanity.com/paper/1706.03466/) (2017)
- [Evolving Deep Neural Networks](https://mlsanity.com/paper/1703.00548/) (2017)
- [Meta-Learning of Neural Architectures for Few-Shot Learning](https://mlsanity.com/paper/1911.11090/) (2019)

## Built on by (most cited first, 30 of 72)

- [Surgical Fine-Tuning Improves Adaptation to Distribution Shifts](https://mlsanity.com/paper/2210.11466/) (2022)
- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuning](https://mlsanity.com/paper/2306.07967/) (2023)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning](https://mlsanity.com/paper/2206.06122/) (2022)
- [No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence](https://mlsanity.com/paper/2207.11680/) (2022)
- [LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging](https://mlsanity.com/paper/2410.17146/) (2024)
- [Trainable Projected Gradient Method for Robust Fine-tuning](https://mlsanity.com/paper/2303.10720/) (2023)
- [SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classification](https://mlsanity.com/paper/2211.16191/) (2022)
- [Few-Shot Classification with Contrastive Learning](https://mlsanity.com/paper/2209.08224/) (2022)
- [Reassessing Layer Pruning in LLMs: New Insights and Methods](https://mlsanity.com/paper/2411.15558/) (2024)
- [Adaptive Test-Time Personalization for Federated Learning](https://mlsanity.com/paper/2310.18816/) (2023)
- [PromptST: Prompt-Enhanced Spatio-Temporal Multi-Attribute Prediction](https://mlsanity.com/paper/2309.09500/) (2023)
- [Zero-Shot Object-Centric Representation Learning](https://mlsanity.com/paper/2408.09162/) (2024)
- [Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How](https://mlsanity.com/paper/2306.03828/) (2023)
- [Adaptive Poincar'e Point to Set Distance for Few-Shot Classification](https://mlsanity.com/paper/2112.01719/) (2021)
- [Comparison of fine-tuning strategies for transfer learning in medical image classification](https://mlsanity.com/paper/2406.10050/) (2024)
- [DETA: Denoised Task Adaptation for Few-Shot Learning](https://mlsanity.com/paper/2303.06315/) (2023)
- [Disentangled Feature Representation for Few-shot Image Classification](https://mlsanity.com/paper/2109.12548/) (2021)
- [MetaNODE: Prototype Optimization as a Neural ODE for Few-Shot Learning](https://mlsanity.com/paper/2103.14341/) (2021)
- [Label Hallucination for Few-Shot Classification](https://mlsanity.com/paper/2112.03340/) (2021)
- [Exploring Parameter-Efficient Fine-Tuning to Enable Foundation Models in Federated Learning](https://mlsanity.com/paper/2210.01708/) (2022)
- [Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensing](https://mlsanity.com/paper/2310.12452/) (2023)
- [Large Convolutional Model Tuning via Filter Subspace](https://mlsanity.com/paper/2403.00269/) (2024)
- [CAD: Co-Adapting Discriminative Features for Improved Few-Shot Classification](https://mlsanity.com/paper/2203.13465/) (2022)
- [PRO-VPT: Distribution-Adaptive Visual Prompt Tuning via Prompt Relocation](https://mlsanity.com/paper/2503.06901/) (2025)
- [p-Meta: Towards On-device Deep Model Adaptation](https://mlsanity.com/paper/2206.12705/) (2022)
- [A Recent Survey of Heterogeneous Transfer Learning](https://mlsanity.com/paper/2310.08459/) (2023)
- [Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformers](https://mlsanity.com/paper/2312.15681/) (2023)
- [Enhancing the De-identification of Personally Identifiable Information in Educational Data](https://mlsanity.com/paper/2501.09765/) (2025)

## Abstract (arXiv)

The goal of few-shot learning is to learn a classifier that can recognize unseen classes from limited support data with labels. A common practice for this task is to train a model on the base set first and then transfer to novel classes through fine-tuning (Here fine-tuning procedure is defined as transferring knowledge from base to novel data, i.e. learning to transfer in few-shot scenario.) or meta-learning. However, as the base classes have no overlap to the novel set, simply transferring whole knowledge from base data is not an optimal solution since some knowledge in the base model may be biased or even harmful to the novel class. In this paper, we propose to transfer partial knowledge by freezing or fine-tuning particular layer(s) in the base model. Specifically, layers will be imposed different learning rates if they are chosen to be fine-tuned, to control the extent of preserved transferability. To determine which layers to be recast and what values of learning rates for them, we introduce an evolutionary search based method that is efficient to simultaneously locate the target layers and determine their individual learning rates. We conduct extensive experiments on CUB and mini-ImageNet to demonstrate the effectiveness of our proposed method. It achieves the state-of-the-art performance on both meta-learning and non-meta based frameworks. Furthermore, we extend our method to the conventional pre-training + fine-tuning paradigm and obtain consistent improvement.