mlsanityData to 7 Oct 2026

Few and Fewer: Learning Better from Few Examples Using Fewer Base Classes

arXiv 2401.15834 ↗First posted 29 Jan 2024cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

When training data is scarce, it is common to make use of a feature extractor that has been pre-trained on a large base dataset, either by fine-tuning its parameters on the ``target'' dataset or by directly adopting its representation as features for a simple classifier. Fine-tuning is ineffective for few-shot learning, since the target dataset contains only a handful of examples. However, directly adopting the features without fine-tuning relies on the base and target distributions being similar enough that these features achieve separability and generalization. This paper investigates whether better features for the target dataset can be obtained by training on fewer base classes, seeking to identify a more useful base dataset for a given task.We consider cross-domain few-shot image classification in eight different domains from Meta-Dataset and entertain multiple real-world settings (domain-informed, task-informed and uninformed) where progressively less detail is known about the target task. To our knowledge, this is the first demonstration that fine-tuning on a subset of carefully selected base classes can significantly improve few-shot learning. Our contributions are simple and intuitive methods that can be implemented in any few-shot solution. We also give insights into the conditions in which these solutions are likely to provide a boost in accuracy. We release the code to reproduce all experiments from this paper on GitHub. https://github.com/RafLaf/Few-and-Fewer.git

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 29 Jan 2024.

Builds on

42 of 59 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Segment Anythingcited by 8,158
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  8. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2013Describing Textures in the Wildcited by 2,110
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2017Learning multiple visual domains with residual adapterscited by 553
30 more
  1. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  2. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  3. 2018Meta-learning with differentiable closed-form solverscited by 395
  4. 2017Few-Shot Learning with Graph Neural Networkscited by 324
  5. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  6. 2019Cross Attention Network for Few-shot Classificationcited by 218
  7. 2019Task2Vec: Task Embedding for Meta-Learningcited by 197
  8. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  9. 2018Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learningcited by 174
  10. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  11. 2022RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rankcited by 145
  12. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  13. 2019Improved Few-Shot Visual Classificationcited by 114
  14. 2016Learning feed-forward one-shot learnerscited by 93
  15. 2019TapNet: Neural Network Augmented with Task-Adaptive Projection for Few-Shot Learningcited by 90
  16. 2021Cross-domain Few-shot Learning with Task-specific Adapterscited by 79
  17. 2019When Does Self-supervision Improve Few-shot Learning?cited by 78
  18. 2021Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learningcited by 72
  19. 2021Mining Latent Classes for Few-shot Segmentationcited by 69
  20. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  21. 2021Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Datacited by 49
  22. 2017Borrowing Treasures from the Wealthy: Deep Transfer Learning through Selective Joint Fine-tuningcited by 48
  23. 2023A Closer Look at Few-shot Classification Againcited by 46
  24. 2022Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficultycited by 37
  25. 2021Improved Fine-Tuning by Better Leveraging Pre-Training Datacited by 36
  26. 2020Domain Generalizer: A Few-shot Meta Learning Framework for Domain Generalization in Medical Imagingcited by 18
  27. 2022EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredientscited by 15
  28. 2021Bridging Few-Shot Learning and Adaptation: New Challenges of Support-Query Shiftcited by 9
  29. 2021Supervised Momentum Contrastive Learning for Few-Shot Classificationcited by 6
  30. 2022A Statistical Model for Predicting Generalization in Few-Shot Classificationcited by 2

Built on by

Not cited yet.