# How to fine-tune deep neural networks in few-shot learning?

arXiv: https://arxiv.org/abs/2012.00204 · HTML: https://mlsanity.com/paper/2012.00204/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2020.

## Built on by (most cited first, 4 of 4)

- [Reassessing Layer Pruning in LLMs: New Insights and Methods](https://mlsanity.com/paper/2411.15558/) (2024)
- [Few-Shot Learning in Video and 3D Object Detection: A Survey](https://mlsanity.com/paper/2507.17079/) (2025)
- [On Fine-Tuned Deep Features for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2210.14083/) (2022)
- [Application of deep learning approaches for medieval historical documents transcription](https://mlsanity.com/paper/2512.18865/) (2025)

## Abstract (arXiv)

Deep learning has been widely used in data-intensive applications. However, training a deep neural network often requires a large data set. When there is not enough data available for training, the performance of deep learning models is even worse than that of shallow networks. It has been proved that few-shot learning can generalize to new tasks with few training samples. Fine-tuning of a deep model is simple and effective few-shot learning method. However, how to fine-tune deep learning models (fine-tune convolution layer or BN layer?) still lack deep investigation. Hence, we study how to fine-tune deep models through experimental comparison in this paper. Furthermore, the weight of the models is analyzed to verify the feasibility of the fine-tuning method.