mlsanityData to 13 Oct 2026

Free Lunch for Few-shot Learning: Distribution Calibration

arXiv 2101.06395 ↗First posted 16 Jan 2021cs.LG cs.CVcited by 165

Not on the map. Why?

Abstract (arXiv)

Learning from a limited number of samples is challenging since the learned model can easily become overfitted based on the biased distribution formed by only a few training examples. In this paper, we calibrate the distribution of these few-sample classes by transferring statistics from the classes with sufficient examples, then an adequate number of examples can be sampled from the calibrated distribution to expand the inputs to the classifier. We assume every dimension in the feature representation follows a Gaussian distribution so that the mean and the variance of the distribution can borrow from that of similar classes whose statistics are better estimated with an adequate number of samples. Our method can be built on top of off-the-shelf pretrained feature extractors and classification models without extra parameters. We show that a simple logistic regression classifier trained using the features sampled from our calibrated distribution can outperform the state-of-the-art accuracy on two datasets (~5% improvement on miniImageNet compared to the next best). The visualization of these generated features demonstrates that our calibrated distribution is an accurate estimation.

Key facts. Cited by 165 papers in the map (data through 13 Oct 2026). Builds on 21 of its 37 references that are in the map. First posted on arXiv 16 Jan 2021.

Builds on

21 of 37 references.

References · 21

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,476
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,870
  3. 2016Wide Residual Networkscited by 3,063
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,011
  5. 2016Matching Networks for One Shot Learningcited by 2,251
  6. 2019A Closer Look at Few-shot Classificationcited by 683
  7. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  8. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  9. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 378
  10. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  11. 2018Low-Shot Learning from Imaginary Datacited by 181
  12. 2019Charting the Right Manifold: Manifold Mixup for Few-shot Learningcited by 135
9 more
  1. 2020Negative Margin Matters: Understanding Margin in Few-shot Classificationcited by 129
  2. 2020Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspectivecited by 120
  3. 2019Implicit Semantic Data Augmentation for Deep Networkscited by 106
  4. 2018Delta-encoder: an effective sample synthesis method for few-shot object recognitioncited by 89
  5. 2019An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learningcited by 38
  6. 2018Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networkscited by 35
  7. 2019Learning to Propagate for Graph Meta-Learningcited by 29
  8. 2019Assume, Augment and Learn: Unsupervised Few-Shot Meta-Learning via Random Labels and Data Augmentationcited by 24
  9. 2019Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot Learning on Category Graphcited by 15

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 28; 2022: 60; 2023: 41; 2024: 24; 2025: 8; 2026: 4.

165 papers.

Sort:

By area:

  1. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 163
  2. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  3. 2022Registration based Few-Shot Anomaly Detectioncited by 133
  4. 2022Dataset Pruning: Reducing Training Data by Examining Generalization Influencecited by 132
  5. 2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
  6. 2021PartImageNet: A Large, High-Quality Dataset of Partscited by 107
  7. 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86
  8. 2021Sample Selection with Uncertainty of Losses for Learning with Noisy Labelscited by 79
  9. 2022Empowering Graph Representation Learning with Test-Time Graph Transformationcited by 67
  10. 2023Uncertainty Estimation by Fisher Information-based Evidential Deep Learningcited by 58
  11. 2024A Hard-to-Beat Baseline for Training-free CLIP-based Adaptationcited by 53
  12. 2023Few-Shot Class-Incremental Learning via Training-Free Prototype Calibrationcited by 46
38 more
  1. 2022Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Modelscited by 40
  2. 2022Generating Representative Samples for Few-Shot Classificationcited by 39
  3. 2021Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Networkcited by 30
  4. 2022Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learningcited by 27
  5. 2022TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibrationcited by 25
  6. 2023Supervised Masked Knowledge Distillation for Few-Shot Transformerscited by 23
  7. 2022Inductive and Transductive Few-Shot Video Classification via Appearance and Temporal Alignmentscited by 22
  8. 2022Delving into Semantic Scale Imbalancecited by 22
  9. 2021LibFewShot: A Comprehensive Library for Few-shot Learningcited by 21
  10. 2022When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration Methodcited by 21
  11. 2022Channel Importance Matters in Few-Shot Image Classificationcited by 20
  12. 2022Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroidcited by 20
  13. 2023Does Few-shot Learning Suffer from Backdoor Attacks?cited by 20
  14. 2022Self-Supervision Can Be a Good Few-Shot Learnercited by 19
  15. 2021ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learningcited by 18
  16. 2021Objects in Semantic Topologycited by 18
  17. 2023Learning to Learn from APIs: Black-Box Data-Free Meta-Learningcited by 18
  18. 2021TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptationcited by 17
  19. 2021FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understandingcited by 17
  20. 2023VNE: An Effective Method for Improving Deep Representation by Manipulating Eigenvalue Distributioncited by 17
  21. 2023ContrastNet: A Contrastive Learning Framework for Few-Shot Text Classificationcited by 17
  22. 2022EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredientscited by 16
  23. 2022Towards better understanding and better generalization of few-shot classification in histology images with contrastive learningcited by 16
  24. 2023ProtoDiff: Learning to Learn Prototypical Networks by Task-Guided Diffusioncited by 16
  25. 2021A Closer Look at Prototype Classifier for Few-shot Image Classificationcited by 15
  26. 2023Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalizationcited by 15
  27. 2024Revisiting Essential and Nonessential Settings of Evidential Deep Learningcited by 15
  28. 2021Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learningcited by 14
  29. 2022Adaptive Distribution Calibration for Few-Shot Learning with Hierarchical Optimal Transportcited by 14
  30. 2021Label-Aware Distribution Calibration for Long-tailed Classificationcited by 13
  31. 2024Parametric Feature Transfer: One-shot Federated Learning with Foundation Modelscited by 13
  32. 2021Disentangled Feature Representation for Few-shot Image Classificationcited by 12
  33. 2022Learnable Distribution Calibration for Few-Shot Class-Incremental Learningcited by 12
  34. 2021MetaNODE: Prototype Optimization as a Neural ODE for Few-Shot Learningcited by 11
  35. 2022Multi-Faceted Distillation of Base-Novel Commonality for Few-shot Object Detectioncited by 11
  36. 2023Wrapped Cauchy Distributed Angular Softmax for Long-Tailed Visual Recognitioncited by 11
  37. 2024CrossGLG: LLM Guides One-shot Skeleton-based 3D Action Recognition in a Cross-level Mannercited by 11
  38. 2024A Comprehensive Review of Few-shot Action Recognitioncited by 11

50 most cited of 165 All 165 (JSON)