mlsanityData to 14 Oct 2026

Are We Really Doing Few-Shot Learning? A Critical Examination of Pre-Training Assumptions

arXiv 2609.10851 ↗First posted 9 Sep 2026cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Few-shot learning is commonly evaluated under protocols that pre-train a model on a large auxiliary set whose classes are disjoint from the target episodes yet drawn from the same visual domain. This paper examines whether such protocols truly reflect low-data learning. We systematically compare no pre-training, class-disjoint in-domain pre-training, supervised out-of-domain pre-training, and label-free out-of-domain pre-training across eight datasets, three few-shot architectures, and multiple way-shot settings. Our results show that class disjointness alone is insufficient to remove the influence of target-domain data. In-domain pre-training improves over no pre-training by 33.41 percentage points on average, whereas supervised out-of-domain pre-training yields 23.75 percentage points, revealing a 9.66-point optimistic bias associated with domain overlap. Although out-of-domain pre-training is more realistic in applications where target-domain data are scarce, its effectiveness depends strongly on the compatibility between source and target domains. We further show that labeled source data are not strictly required, with an augmentation-based label-free strategy reaching an average gain of 27.71 percentage points and closely matching supervised out-of-domain pre-training at 27.97 percentage points. Finally, we introduce a descriptor-based source-selection strategy that estimates source-domain suitability before pre-training, reaching a median gap of only 1.37 percentage points to oracle selection. These findings highlight the need to move beyond in-domain pre-training as the default few-shot evaluation protocol, since it can overestimate performance in realistic scenarios where target-domain data are scarce.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 73 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

39 of 73 references.

References · 39

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  5. 2016Matching Networks for One Shot Learningcited by 2,255
  6. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  7. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
  8. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 690
  9. 2019A Closer Look at Few-shot Classificationcited by 684
  10. 2022Memory-Based Model Editing at Scalecited by 447
  11. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  12. 2018Meta-learning with differentiable closed-form solverscited by 397
27 more
  1. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
  2. 2019A Baseline for Few-Shot Image Classificationcited by 298
  3. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 294
  4. 2018Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoderscited by 264
  5. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 195
  6. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 163
  7. 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 126
  8. 2020Adversarial Feature Hallucination Networks for Few-Shot Learningcited by 110
  9. 2019Prototype Rectification for Few-Shot Learningcited by 97
  10. 2020Laplacian Regularized Few-Shot Learningcited by 92
  11. 2022Learning from Few Examples: A Summary of Approaches to Few-Shot Learningcited by 86
  12. 2020Leveraging the Feature Distribution in Transfer-based Few-Shot Learningcited by 74
  13. 2023Advances and Challenges in Meta-Learning: A Technical Reviewcited by 65
  14. 2021Cross-Domain Few-Shot Classification via Adversarial Task Augmentationcited by 63
  15. 2021Recurrent Mask Refinement for Few-Shot Medical Image Segmentationcited by 63
  16. 2021Boosting the Generalization Capability in Cross-Domain Few-shot Learning via Noise-enhanced Supervised Autoencodercited by 51
  17. 2021Meta-FDMixup: Cross-Domain Few-Shot Learning Guided by Labeled Target Datacited by 48
  18. 2020On Episodes, Prototypical Networks, and Few-shot Learningcited by 46
  19. 2023A Closer Look at Few-shot Classification Againcited by 46
  20. 2020Domain-Adaptive Few-Shot Learningcited by 40
  21. 2020Speech SIMCLR: Combining Contrastive and Reconstruction Objective for Self-supervised Speech Representation Learningcited by 34
  22. 2021Ranking Distance Calibration for Cross-Domain Few-Shot Learningcited by 27
  23. 2023Meta-learning approaches for few-shot learning: A survey of recent advancescited by 25
  24. 2022Universal Representations: A Unified Look at Multiple Task and Domain Learningcited by 23
  25. 2023Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Surveycited by 14
  26. 2021Few-Shot and Continual Learning with Attentive Independent Mechanismscited by 13
  27. 2024Few-shot learning for COVID-19 Chest X-Ray Classification with Imbalanced Data: An Inter vs. Intra Domain Studycited by 3

Built on by

Not cited yet.