mlsanityData to 7 Oct 2026

Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference

arXiv 2204.07305 ↗First posted 15 Apr 2022cs.CV cs.LGcited by 123

Not on the map. Why?

Abstract (arXiv)

Few-shot learning (FSL) is an important and topical problem in computer vision that has motivated extensive research into numerous methods spanning from sophisticated meta-learning methods to simple transfer learning baselines. We seek to push the limits of a simple-but-effective pipeline for more realistic and practical settings of few-shot image classification. To this end, we explore few-shot learning from the perspective of neural network architecture, as well as a three stage pipeline of network updates under different data supplies, where unsupervised external data is considered for pre-training, base categories are used to simulate few-shot tasks for meta-training, and the scarcely labelled data of an novel task is taken for fine-tuning. We investigate questions such as: (1) How pre-training on external data benefits FSL? (2) How state-of-the-art transformer architectures can be exploited? and (3) How fine-tuning mitigates domain shift? Ultimately, we show that a simple transformer-based pipeline yields surprisingly good performance on standard benchmarks such as Mini-ImageNet, CIFAR-FS, CDFSL and Meta-Dataset. Our code and demo are available at https://hushell.github.io/pmf.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 58 of its 72 references that are in the map. First posted on arXiv 15 Apr 2022.

Builds on

58 of 72 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  12. 2017Prototypical Networks for Few-shot Learningcited by 3,001
38 more
  1. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  2. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  3. 2016Matching Networks for One Shot Learningcited by 2,246
  4. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  5. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  6. 2014How transferable are features in deep neural networks?cited by 1,216
  7. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  8. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  9. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  10. 2019A Closer Look at Few-shot Classificationcited by 679
  11. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
  12. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  13. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  14. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  15. 2018Meta-learning with differentiable closed-form solverscited by 395
  16. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  17. 2019A Baseline for Few-Shot Image Classificationcited by 298
  18. 2020How Well Do Self-Supervised Models Transfer?cited by 213
  19. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 208
  20. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  21. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  22. 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
  23. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  24. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
  25. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  26. 2019Charting the Right Manifold: Manifold Mixup for Few-shot Learningcited by 135
  27. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  28. 2016Learning feed-forward one-shot learnerscited by 93
  29. 2019Learning to Self-Train for Semi-Supervised Few-Shot Classificationcited by 83
  30. 2020Embedding Propagation: Smoother Manifold for Few-Shot Classificationcited by 79
  31. 2020Meta-Learning with Adaptive Hyperparameterscited by 79
  32. 2020Self-training for Few-shot Transfer Across Extreme Task Differencescited by 75
  33. 2020Leveraging the Feature Distribution in Transfer-based Few-Shot Learningcited by 72
  34. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  35. 2021Learning a Universal Template for Few-shot Dataset Generalizationcited by 61
  36. 2021Universal Representation Learning from Multiple Domains for Few-shot Classificationcited by 56
  37. 2021Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learningcited by 49
  38. 2020Empirical Bayes Transductive Meta-Learning with Synthetic Gradientscited by 48

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 44; 2024: 34; 2025: 23; 2026: 12.

123 papers.

Sort:

By area:

  1. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  2. 2023A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervisioncited by 78
  3. 2023StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learningcited by 53
  4. 2023A Closer Look at Few-shot Classification Againcited by 46
  5. 2024Cross-Domain Few-Shot Object Detection via Enhanced Open-Set Object Detectorcited by 43
  6. 2023Improving neural network representations using human similarity judgmentscited by 38
  7. 2023Strong Baselines for Parameter Efficient Few-Shot Fine-tuningcited by 36
  8. 2022SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Modelscited by 34
  9. 2022SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classificationcited by 34
  10. 2024Transductive Zero-Shot and Few-Shot CLIPcited by 31
  11. 2023MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognitioncited by 26
  12. 2024Flatten Long-Range Loss Landscapes for Cross-Domain Few-Shot Learningcited by 26
38 more
  1. 2023Context-Aware Meta-Learningcited by 20
  2. 2023Exploring Incompatible Knowledge Transfer in Few-shot Image Generationcited by 19
  3. 2023Visual and Textual Prior Guided Mask Assemble for Few-Shot Segmentation and Beyondcited by 19
  4. 2023TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edgecited by 18
  5. 2023Exploring Efficient Few-shot Adaptation for Vision Transformerscited by 16
  6. 2025Reliable Few-shot Learning under Dual Noisescited by 16
  7. 2023De-Diffusion Makes Text a Strong Cross-Modal Interfacecited by 14
  8. 2023DETA: Denoised Task Adaptation for Few-Shot Learningcited by 13
  9. 2023Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentationcited by 12
  10. 2024Combining Observational Data and Language for Species Range Estimationcited by 12
  11. 2022Generalization in Neural Networks: A Broad Surveycited by 11
  12. 2023Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-shot Learning with Hyperspherical Embeddingscited by 11
  13. 2023Meta-Learning with a Geometry-Adaptive Preconditionercited by 11
  14. 2023On the Importance of Spatial Relations for Few-shot Action Recognitioncited by 11
  15. 2023LFS-GAN: Lifelong Few-Shot Image Generationcited by 11
  16. 2023LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platformscited by 11
  17. 2025Foundation Models for Bioacoustics -- a Comparative Reviewcited by 11
  18. 2022Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classificationcited by 10
  19. 2022Cap2Aug: Caption guided Image to Image data Augmentationcited by 10
  20. 2023Optimizing PatchCore for Few/many-shot Anomaly Detectioncited by 10
  21. 2024Prompt as Free Lunch: Enhancing Diversity in Source-Free Cross-domain Few-shot Learning through Semantic-Guided Promptingcited by 10
  22. 2022Few-shot Metric Learning: Online Adaptation of Embedding for Retrievalcited by 8
  23. 2023Masked Autoencoding Does Not Help Natural Language Supervision at Scalecited by 7
  24. 2023MVP: Meta Visual Prompt Tuning for Few-Shot Remote Sensing Image Scene Classificationcited by 7
  25. 2024Cross-Domain Few-Shot Learning via Adaptive Transformer Networkscited by 7
  26. 2024Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learningcited by 7
  27. 2024Making Large Vision Language Models to be Good Few-shot Learnerscited by 7
  28. 2024Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio Sensorscited by 7
  29. 2025DiTASK: Multi-Task Fine-Tuning with Diffeomorphic Transformationscited by 7
  30. 2025Don't throw the baby out with the bathwater: How and why deep learning for ARCcited by 7
  31. 2023BDC-Adapter: Brownian Distance Covariance for Better Vision-Language Reasoningcited by 6
  32. 2023HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedbackcited by 6
  33. 2024BECLR: Batch Enhanced Contrastive Few-Shot Learningcited by 6
  34. 2023Instance-based Max-margin for Practical Few-shot Recognitioncited by 5
  35. 2023Improved Region Proposal Network for Enhanced Few-Shot Object Detectioncited by 5
  36. 2024Learning to Obstruct Few-Shot Image Classification over Restricted Classescited by 5
  37. 2026Reclaiming Lost Text Layers for Source-Free Cross-Domain Few-Shot Learningcited by 5
  38. 2022Enhancing Few-shot Image Classification with Cosine Transformercited by 4

50 most cited of 123 All 123 (JSON)