mlsanityData to 7 Oct 2026

Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning

arXiv 2110.09446 ↗First posted 18 Oct 2021cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Few-shot classification is a challenging problem due to the uncertainty caused by using few labelled samples. In the past few years, many methods have been proposed with the common aim of transferring knowledge acquired on a previously solved task, what is often achieved by using a pretrained feature extractor. Following this vein, in this paper we propose a novel transfer-based method which aims at processing the feature vectors so that they become closer to Gaussian-like distributions, resulting in increased accuracy. In the case of transductive few-shot learning where unlabelled test samples are available during training, we also introduce an optimal-transport inspired algorithm to boost even further the achieved performance. Using standardized vision benchmarks, we show the ability of the proposed methodology to achieve state-of-the-art accuracy with various datasets, backbone architectures and few-shot settings.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 50 of its 70 references that are in the map. First posted on arXiv 18 Oct 2021.

Builds on

50 of 70 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2016Wide Residual Networkscited by 3,057
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  8. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2019A Closer Look at Few-shot Classificationcited by 679
  11. 2015Towards Open Set Deep Networkscited by 533
  12. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
38 more
  1. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  2. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  3. 2018Meta-learning with differentiable closed-form solverscited by 395
  4. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
  5. 2018How to train your MAMLcited by 306
  6. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 293
  7. 2020Frustratingly Simple Few-Shot Object Detectioncited by 280
  8. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 208
  9. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  10. 2021Free Lunch for Few-shot Learning: Distribution Calibrationcited by 165
  11. 2019Charting the Right Manifold: Manifold Mixup for Few-shot Learningcited by 135
  12. 2020Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotationcited by 128
  13. 2019Prototype Rectification for Few-Shot Learningcited by 96
  14. 2020Laplacian Regularized Few-Shot Learningcited by 90
  15. 2020Embedding Propagation: Smoother Manifold for Few-Shot Classificationcited by 79
  16. 2021Relational Embedding for Few-Shot Classificationcited by 74
  17. 2020Leveraging the Feature Distribution in Transfer-based Few-Shot Learningcited by 72
  18. 2020Self-Distillation as Instance-Specific Label Smoothingcited by 71
  19. 2021Recurrent Mask Refinement for Few-Shot Medical Image Segmentationcited by 63
  20. 2019Few-shot Text Classification with Distributional Signaturescited by 58
  21. 2019Learning to Few-Shot Learn Across Diverse Natural Language Classification Taskscited by 57
  22. 2019Image Deformation Meta-Networks for One-Shot Learningcited by 56
  23. 2021Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learningcited by 56
  24. 2020Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Taskscited by 51
  25. 2020Few-Shot Open-Set Recognition using Meta-Learningcited by 50
  26. 2019Few-Shot Learning via Saliency-guided Hallucination of Samplescited by 49
  27. 2020Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embeddingcited by 48
  28. 2020Transductive Information Maximization For Few-Shot Learningcited by 44
  29. 2020Self-supervised Knowledge Distillation for Few-shot Learningcited by 41
  30. 2020TAFSSL: Task-Adaptive Feature Sub-Space Learning for few-shot classificationcited by 30
  31. 2021Few-shot Open-set Recognition by Transformation Consistencycited by 29
  32. 2019A Two-Stage Approach to Few-Shot Learning for Image Recognitioncited by 18
  33. 2020Meta-Learned Confidence for Few-shot Learningcited by 9
  34. 2021Sill-Net: Feature Augmentation with Separated Illumination Representationcited by 7
  35. 2021Transfer learning based few-shot classification using optimal transport mapping from preprocessed latent space of backbone neural networkcited by 6
  36. 2020A Study of Few-Shot Audio Classificationcited by 5
  37. 2015Approximate Fisher Kernels of non-iid Image Models for Image Categorizationcited by 2
  38. 2017Improving Accuracy of Nonparametric Transfer Learning via Vector Segmentationcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 11; 2023: 2; 2024: 0; 2025: 1.

14 papers.

Sort:

By area:

  1. 2022SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classificationcited by 34
  2. 2022Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?cited by 31
  3. 2023Visual and Textual Prior Guided Mask Assemble for Few-Shot Segmentation and Beyondcited by 19
  4. 2022EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredientscited by 15
  5. 2022Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusionscited by 13
  6. 2022Generalization in Neural Networks: A Broad Surveycited by 11
  7. 2022Adaptive Dimension Reduction and Variational Inference for Transductive Few-Shot Classificationcited by 10
  8. 2022Transductive Decoupled Variational Inference for Few-Shot Classificationcited by 7
  9. 2022Dynamic Sub-Cluster-Aware Network for Few-Shot Skin Disease Classificationcited by 3
  10. 2022A Statistical Model for Predicting Generalization in Few-Shot Classificationcited by 2
  11. 2022Reconciliation of Pre-trained Models and Prototypical Neural Networks in Few-shot Named Entity Recognitioncited by 1
  12. 2022Class-Specific Channel Attention for Few-Shot Learningcited by 0
2 more
  1. 2023Few-shot Image Classification based on Gradual Machine Learningcited by 0
  2. 2025ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluationcited by 0