# Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning

arXiv: https://arxiv.org/abs/2110.09446 · HTML: https://mlsanity.com/paper/2110.09446/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 70 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 30 of 50)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Frustratingly Simple Few-Shot Object Detection](https://mlsanity.com/paper/2003.06957/) (2020)
- [Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning](https://mlsanity.com/paper/1805.10002/) (2018)
- [SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning](https://mlsanity.com/paper/1911.04623/) (2019)
- [Free Lunch for Few-shot Learning: Distribution Calibration](https://mlsanity.com/paper/2101.06395/) (2021)
- [Charting the Right Manifold: Manifold Mixup for Few-shot Learning](https://mlsanity.com/paper/1907.12087/) (2019)
- [Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation](https://mlsanity.com/paper/2007.09886/) (2020)
- [Prototype Rectification for Few-Shot Learning](https://mlsanity.com/paper/1911.10713/) (2019)
- [Laplacian Regularized Few-Shot Learning](https://mlsanity.com/paper/2006.15486/) (2020)
- [Embedding Propagation: Smoother Manifold for Few-Shot Classification](https://mlsanity.com/paper/2003.04151/) (2020)
- [Leveraging the Feature Distribution in Transfer-based Few-Shot Learning](https://mlsanity.com/paper/2006.03806/) (2020)
- [Relational Embedding for Few-Shot Classification](https://mlsanity.com/paper/2108.09666/) (2021)
- [Self-Distillation as Instance-Specific Label Smoothing](https://mlsanity.com/paper/2006.05065/) (2020)

## Built on by (most cited first, 14 of 14)

- [SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classification](https://mlsanity.com/paper/2211.16191/) (2022)
- [Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?](https://mlsanity.com/paper/2203.17271/) (2022)
- [Visual and Textual Prior Guided Mask Assemble for Few-Shot Segmentation and Beyond](https://mlsanity.com/paper/2308.07539/) (2023)
- [EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients](https://mlsanity.com/paper/2201.09699/) (2022)
- [Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions](https://mlsanity.com/paper/2202.11423/) (2022)
- [Generalization in Neural Networks: A Broad Survey](https://mlsanity.com/paper/2209.01610/) (2022)
- [Adaptive Dimension Reduction and Variational Inference for Transductive Few-Shot Classification](https://mlsanity.com/paper/2209.08527/) (2022)
- [Transductive Decoupled Variational Inference for Few-Shot Classification](https://mlsanity.com/paper/2208.10559/) (2022)
- [Dynamic Sub-Cluster-Aware Network for Few-Shot Skin Disease Classification](https://mlsanity.com/paper/2207.01072/) (2022)
- [A Statistical Model for Predicting Generalization in Few-Shot Classification](https://mlsanity.com/paper/2212.06461/) (2022)
- [Reconciliation of Pre-trained Models and Prototypical Neural Networks in Few-shot Named Entity Recognition](https://mlsanity.com/paper/2211.03270/) (2022)
- [Class-Specific Channel Attention for Few-Shot Learning](https://mlsanity.com/paper/2209.01332/) (2022)
- [Few-shot Image Classification based on Gradual Machine Learning](https://mlsanity.com/paper/2307.15524/) (2023)
- [ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluation](https://mlsanity.com/paper/2507.09299/) (2025)

## Abstract (arXiv)

Few-shot classification is a challenging problem due to the uncertainty caused by using few labelled samples. In the past few years, many methods have been proposed with the common aim of transferring knowledge acquired on a previously solved task, what is often achieved by using a pretrained feature extractor. Following this vein, in this paper we propose a novel transfer-based method which aims at processing the feature vectors so that they become closer to Gaussian-like distributions, resulting in increased accuracy. In the case of transductive few-shot learning where unlabelled test samples are available during training, we also introduce an optimal-transport inspired algorithm to boost even further the achieved performance. Using standardized vision benchmarks, we show the ability of the proposed methodology to achieve state-of-the-art accuracy with various datasets, backbone architectures and few-shot settings.