# The Role of Global Labels in Few-Shot Classification and How to Infer Them

arXiv: https://arxiv.org/abs/2108.04055 · HTML: https://mlsanity.com/paper/2108.04055/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 9 Aug 2021.

## Builds on (most cited first, 28 of 28)

- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Bilevel Programming for Hyperparameter Optimization and Meta-Learning](https://mlsanity.com/paper/1806.04910/) (2018)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Charting the Right Manifold: Manifold Mixup for Few-shot Learning](https://mlsanity.com/paper/1907.12087/) (2019)
- [Few-Shot Image Recognition by Predicting Parameters from Activations](https://mlsanity.com/paper/1706.03466/) (2017)
- [Few-Shot Classification with Feature Map Reconstruction Networks](https://mlsanity.com/paper/2012.01506/) (2020)
- [Unsupervised Learning via Meta-Learning](https://mlsanity.com/paper/1810.02334/) (2018)
- [Embedding Propagation: Smoother Manifold for Few-Shot Classification](https://mlsanity.com/paper/2003.04151/) (2020)
- [Few-Shot Learning with Embedded Class Models and Shot-Free Meta Training](https://mlsanity.com/paper/1905.04398/) (2019)
- [The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning](https://mlsanity.com/paper/2008.10857/) (2020)

## Built on by (most cited first, 12 of 12)

- [Inductive and Transductive Few-Shot Video Classification via Appearance and Temporal Alignments](https://mlsanity.com/paper/2207.10785/) (2022)
- [Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-shot Learning with Hyperspherical Embeddings](https://mlsanity.com/paper/2303.09352/) (2023)
- [Lessons learned from the NeurIPS 2021 MetaDL challenge: Backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classification](https://mlsanity.com/paper/2206.08138/) (2022)
- [The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence](https://mlsanity.com/paper/2208.01545/) (2022)
- [Few-shot Action Recognition with Captioning Foundation Models](https://mlsanity.com/paper/2310.10125/) (2023)
- [Is Pre-training Truly Better Than Meta-Learning?](https://mlsanity.com/paper/2306.13841/) (2023)
- [Schedule-Robust Online Continual Learning](https://mlsanity.com/paper/2210.05561/) (2022)
- [Transfer learning for atomistic simulations using GNNs and kernel mean embeddings](https://mlsanity.com/paper/2306.01589/) (2023)
- [PLATINUM: Semi-Supervised Model Agnostic Meta-Learning using Submodular Mutual Information](https://mlsanity.com/paper/2201.12928/) (2022)
- [Modularity in Deep Learning: A Survey](https://mlsanity.com/paper/2310.01154/) (2023)
- [Robust Meta-Representation Learning via Global Label Inference and Classification](https://mlsanity.com/paper/2212.11702/) (2022)
- [Compositional Prototypical Networks for Few-Shot Classification](https://mlsanity.com/paper/2306.06584/) (2023)

## Abstract (arXiv)

Few-shot learning is a central problem in meta-learning, where learners must quickly adapt to new tasks given limited training data. Recently, feature pre-training has become a ubiquitous component in state-of-the-art meta-learning methods and is shown to provide significant performance improvement. However, there is limited theoretical understanding of the connection between pre-training and meta-learning. Further, pre-training requires global labels shared across tasks, which may be unavailable in practice. In this paper, we show why exploiting pre-training is theoretically advantageous for meta-learning, and in particular the critical role of global labels. This motivates us to propose Meta Label Learning (MeLa), a novel meta-learning framework that automatically infers global labels to obtains robust few-shot models. Empirically, we demonstrate that MeLa is competitive with existing methods and provide extensive ablation experiments to highlight its key properties.