mlsanityData to 7 Oct 2026

Rethinking Generalization in Few-Shot Classification

arXiv 2206.07267 ↗First posted 15 Jun 2022cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

Single image-level annotations only correctly describe an often small subset of an image's content, particularly when complex real-world scenes are depicted. While this might be acceptable in many classification scenarios, it poses a significant challenge for applications where the set of classes differs significantly between training and test time. In this paper, we take a closer look at the implications in the context of textitfew-shot learning. Splitting the input samples into patches and encoding these via the help of Vision Transformers allows us to establish semantic correspondences between local regions across images and independent of their respective class. The most informative patch embeddings for the task at hand are then determined as a function of the support set via online optimization at inference time, additionally providing visual interpretability of `textitwhat matters most' in the image. We build on recent advances in unsupervised training of networks via masked image modelling to overcome the lack of fine-grained labels and learn the more general statistical structure of the data while avoiding negative image-level annotation influence, textitaka supervision collapse. Experimental results show the competitiveness of our approach, achieving new state-of-the-art results on four popular few-shot classification benchmarks for 5-shot and 1-shot scenarios.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 51 of its 74 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

51 of 74 references.

References · 51

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2016Matching Networks for One Shot Learningcited by 2,246
  12. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
38 more
  1. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  2. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  3. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  4. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  5. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  6. 2019A Closer Look at Few-shot Classificationcited by 679
  7. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  8. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  9. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  10. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  11. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  12. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  13. 2018Meta-learning with differentiable closed-form solverscited by 395
  14. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 299
  15. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 293
  16. 2019Cross Attention Network for Few-shot Classificationcited by 218
  17. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  18. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  19. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  20. 2019Charting the Right Manifold: Manifold Mixup for Few-shot Learningcited by 135
  21. 2020Few-Shot Classification with Feature Map Reconstruction Networkscited by 116
  22. 2021MST: Masked Self-Supervised Transformer for Visual Representationcited by 114
  23. 2021Efficient Training of Visual Transformers with Small Datasetscited by 107
  24. 2021SiT: Self-supervised vIsion Transformercited by 99
  25. 2019When Does Self-supervision Improve Few-shot Learning?cited by 78
  26. 2021Relational Embedding for Few-Shot Classificationcited by 74
  27. 2021VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learningcited by 56
  28. 2021Pareto Self-Supervised Training for Few-Shot Learningcited by 52
  29. 2021Rectifying the Shortcut Learning of Background for Few-Shot Learningcited by 51
  30. 2022Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classificationcited by 50
  31. 2021Shallow Bayesian Meta Learning for Real-World Few-Shot Recognitioncited by 48
  32. 2021Learning Dynamic Alignment via Meta-filter for Few-shot Learningcited by 41
  33. 2021Partner-Assisted Learning for Few-Shot Image Classificationcited by 36
  34. 2021Binocular Mutual Learning for Improving Few-shot Classificationcited by 26
  35. 2020Mixture-based Feature Space Learning for Few-shot Image Classificationcited by 23
  36. 2021Transductive Few-Shot Classification on the Oblique Manifoldcited by 23
  37. 2021Meta Navigator: Search for a Good Adaptation Policy for Few-shot Learningcited by 19
  38. 2021Adaptive Poincar'e Point to Set Distance for Few-Shot Classificationcited by 15

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 14; 2025: 9; 2026: 2.

36 papers.

Sort:

By area:

  1. 2023Strong Baselines for Parameter Efficient Few-Shot Fine-tuningcited by 36
  2. 2023Supervised Masked Knowledge Distillation for Few-Shot Transformerscited by 23
  3. 2023Simple Semantic-Aided Few-Shot Learningcited by 15
  4. 2023Masking Strategies for Background Bias Removal in Computer Vision Modelscited by 9
  5. 2025Connecting Giants: Synergistic Knowledge Transfer of Large Multimodal Models for Few-Shot Learningcited by 9
  6. 2024Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learningcited by 7
  7. 2025VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learningcited by 7
  8. 2023Mitigating the Effect of Incidental Correlations on Part-based Learningcited by 6
  9. 2024BECLR: Batch Enhanced Contrastive Few-Shot Learningcited by 6
  10. 2023Task-Oriented Channel Attention for Fine-Grained Few-Shot Classificationcited by 4
  11. 2024CLCE: An Approach to Refining Cross-Entropy and Contrastive Learning for Optimized Learning Fusioncited by 4
  12. 2023PrototypeFormer: Learning to Explore Prototype Relationships for Few-shot Image Classificationcited by 3
24 more
  1. 2024Robust Few-Shot Ensemble Learning with Focal Diversity-Based Pruningcited by 3
  2. 2024Learning to Learn with Contrastive Meta-Objectivecited by 3
  3. 2024KNN Transformer with Pyramid Prompts for Few-Shot Learningcited by 3
  4. 2023Contrastive Meta-Learning for Partially Observable Few-Shot Learningcited by 2
  5. 2024Show and Grasp: Few-shot Semantic Segmentation for Robot Grasping through Zero-shot Foundation Modelscited by 2
  6. 2026DVLA-RL: Dual-Level Vision-Language Alignment with Reinforcement Learning Gating for Few-Shot Learningcited by 2
  7. 2024Task Attribute Distance for Few-Shot Learning: Theoretical Analysis and Applicationscited by 1
  8. 2024Intra-task Mutual Attention based Vision Transformer for Few-Shot Learningcited by 1
  9. 2024Class-relevant Patch Embedding Selection for Few-Shot Image Classificationcited by 1
  10. 2024Harnessing the Power of Local Representations for Few-Shot Classificationcited by 1
  11. 2025Slot Attention-based Feature Filtering for Few-Shot Learningcited by 1
  12. 2026Rethinking the Good Enough Embedding for Easy Few-Shot Learningcited by 1
  13. 2023Epistemic Graph: A Plug-And-Play Module For Hybrid Representation Learningcited by 0
  14. 2023Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Workcited by 0
  15. 2023CHIP: Contrastive Hierarchical Image Pretrainingcited by 0
  16. 2024Like Humans to Few-Shot Learning through Knowledge Permeation of Vision and Textcited by 0
  17. 2024Siamese Transformer Networks for Few-shot Image Classificationcited by 0
  18. 2024A Simple Task-aware Contrastive Local Descriptor Selection Strategy for Few-shot Learning between inter class and intra classcited by 0
  19. 2025Brain Inspired Adaptive Memory Dual-Net for Few-Shot Image Classificationcited by 0
  20. 2025ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluationcited by 0
  21. 2025MetaLab: Few-Shot Game Changer for Image Recognitioncited by 0
  22. 2025Color as the Impetus: Transforming Few-Shot Learnercited by 0
  23. 2025Stochastic-based Patch Filtering for Few-Shot Learningcited by 0
  24. 2025BATR-FST: Bi-Level Adaptive Token Refinement for Few-Shot Transformerscited by 0