# Revisiting Fine-tuning for Few-shot Learning

arXiv: https://arxiv.org/abs/1910.00216 · HTML: https://mlsanity.com/paper/1910.00216/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2019.

## Built on by (most cited first, 29 of 29)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Pose for Everything: Towards Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2207.10387/) (2022)
- [Tem-adapter: Adapting Image-Text Pretraining for Video Question Answer](https://mlsanity.com/paper/2308.08414/) (2023)
- [X-Pose: Detecting Any Keypoints](https://mlsanity.com/paper/2310.08530/) (2023)
- [A Graph-Based Approach for Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2311.17891/) (2023)
- [Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey](https://mlsanity.com/paper/2303.08557/) (2023)
- [Open-Vocabulary Animal Keypoint Detection with Semantic-feature Matching](https://mlsanity.com/paper/2310.05056/) (2023)
- [Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensing](https://mlsanity.com/paper/2310.12452/) (2023)
- [Mask-guided Vision Transformer (MG-ViT) for Few-Shot Learning](https://mlsanity.com/paper/2205.09995/) (2022)
- [KARINA: An Efficient Deep Learning Model for Global Weather Forecast](https://mlsanity.com/paper/2403.10555/) (2024)
- [CapeX: Category-Agnostic Pose Estimation from Textual Point Explanation](https://mlsanity.com/paper/2406.00384/) (2024)
- [Generalized Few-Shot Semantic Segmentation: All You Need is Fine-Tuning](https://mlsanity.com/paper/2112.10982/) (2021)
- [Dual-path Frequency Discriminators for Few-shot Anomaly Detection](https://mlsanity.com/paper/2403.04151/) (2024)
- [Meta-Point Learning and Refining for Category-Agnostic Pose Estimation](https://mlsanity.com/paper/2403.13647/) (2024)
- [CRoFT: Robust Fine-Tuning with Concurrent Optimization for OOD Generalization and Open-Set OOD Detection](https://mlsanity.com/paper/2405.16417/) (2024)
- [KptLLM: Unveiling the Power of Large Language Model for Keypoint Comprehension](https://mlsanity.com/paper/2411.01846/) (2024)
- [DAMSL: Domain Agnostic Meta Score-based Learning](https://mlsanity.com/paper/2106.03041/) (2021)
- [SCAPE: A Simple and Strong Category-Agnostic Pose Estimator](https://mlsanity.com/paper/2407.13483/) (2024)
- [Exploring Few-Shot Defect Segmentation in General Industrial Scenarios with Metric Learning and Vision Foundation Models](https://mlsanity.com/paper/2502.01216/) (2025)
- [Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification](https://mlsanity.com/paper/2307.11031/) (2023)
- [Spectral Propagation Graph Network for Few-shot Time Series Classification](https://mlsanity.com/paper/2202.04769/) (2022)
- [Contrastive Meta-Learning for Partially Observable Few-Shot Learning](https://mlsanity.com/paper/2301.13136/) (2023)
- [Dual Adversarial Alignment for Realistic Support-Query Shift Few-shot Learning](https://mlsanity.com/paper/2309.02088/) (2023)
- [Towards Few-Annotation Learning in Computer Vision: Application to Image Classification and Object Detection tasks](https://mlsanity.com/paper/2311.04888/) (2023)
- [Model X-Ray: Detection of Hidden Malware in AI Model Weights using Few Shot Learning](https://mlsanity.com/paper/2409.19310/) (2024)
- [PictSure: Pretraining Embeddings Matters for In-Context Learning Image Classifiers](https://mlsanity.com/paper/2506.14842/) (2025)
- [KptLLM++: Towards Generic Keypoint Comprehension with Large Language Model](https://mlsanity.com/paper/2507.11102/) (2025)
- [Beyond Class Tokens: LLM-guided Dominant Property Mining for Few-shot Classification](https://mlsanity.com/paper/2507.20511/) (2025)
- [FALCON: Few-Shot Adversarial Learning for Cross-Domain Medical Image Segmentation](https://mlsanity.com/paper/2601.01687/) (2026)

## Abstract (arXiv)

Few-shot learning is the process of learning novel classes using only a few examples and it remains a challenging task in machine learning. Many sophisticated few-shot learning algorithms have been proposed based on the notion that networks can easily overfit to novel examples if they are simply fine-tuned using only a few examples. In this study, we show that in the commonly used low-resolution mini-ImageNet dataset, the fine-tuning method achieves higher accuracy than common few-shot learning algorithms in the 1-shot task and nearly the same accuracy as that of the state-of-the-art algorithm in the 5-shot task. We then evaluate our method with more practical tasks, namely the high-resolution single-domain and cross-domain tasks. With both tasks, we show that our method achieves higher accuracy than common few-shot learning algorithms. We further analyze the experimental results and show that: 1) the retraining process can be stabilized by employing a low learning rate, 2) using adaptive gradient optimizers during fine-tuning can increase test accuracy, and 3) test accuracy can be improved by updating the entire network when a large domain-shift exists between base and novel classes.