# Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks

arXiv: https://arxiv.org/abs/2002.06753 · HTML: https://mlsanity.com/paper/2002.06753/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Feb 2020.

## Built on by (most cited first, 30 of 47)

- [Task Residual for Tuning Vision-Language Models](https://mlsanity.com/paper/2211.10277/) (2022)
- [Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer](https://mlsanity.com/paper/2108.03032/) (2021)
- [What makes unlearning hard and what to do about it](https://mlsanity.com/paper/2406.01257/) (2024)
- [Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation](https://mlsanity.com/paper/2106.09017/) (2021)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition](https://mlsanity.com/paper/2101.02833/) (2021)
- [Orthogonal Projection Loss](https://mlsanity.com/paper/2103.14021/) (2021)
- [Reverse Engineering Self-Supervised Learning](https://mlsanity.com/paper/2305.15614/) (2023)
- [Binocular Mutual Learning for Improving Few-shot Classification](https://mlsanity.com/paper/2108.12104/) (2021)
- [Deconfounded Representation Similarity for Comparison of Neural Networks](https://mlsanity.com/paper/2202.00095/) (2022)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [On sensitivity of meta-learning to support data](https://mlsanity.com/paper/2110.13953/) (2021)
- [Uncertainty Herding: One Active Learning Method for All Label Budgets](https://mlsanity.com/paper/2412.20644/) (2024)
- [A Representation Learning Perspective on the Importance of Train-Validation Splitting in Meta-Learning](https://mlsanity.com/paper/2106.15615/) (2021)
- [Meta-learning with negative learning rates](https://mlsanity.com/paper/2102.00940/) (2021)
- [Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practice](https://mlsanity.com/paper/2211.07206/) (2022)
- [The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence](https://mlsanity.com/paper/2208.01545/) (2022)
- [Open-Set Likelihood Maximization for Few-Shot Learning](https://mlsanity.com/paper/2301.08390/) (2023)
- [Bridging Few-Shot Learning and Adaptation: New Challenges of Support-Query Shift](https://mlsanity.com/paper/2105.11804/) (2021)
- [MAML is a Noisy Contrastive Learner in Classification](https://mlsanity.com/paper/2106.15367/) (2021)
- [FS-BAN: Born-Again Networks for Domain Generalization Few-Shot Classification](https://mlsanity.com/paper/2208.10930/) (2022)
- [Generalization Bounds for Few-Shot Transfer Learning with Pretrained Classifiers](https://mlsanity.com/paper/2212.12532/) (2022)
- [Contrastive Representation Shaping for LLM Unlearning](https://mlsanity.com/paper/2601.22028/) (2026)
- [A Channel Coding Benchmark for Meta-Learning](https://mlsanity.com/paper/2107.07579/) (2021)
- [Is Pre-training Truly Better Than Meta-Learning?](https://mlsanity.com/paper/2306.13841/) (2023)
- [BTBR: A Bayesian-Theory-Driven Probabilistic-Fuzzy Framework for Implicit Bias Removal in Large Language Models](https://mlsanity.com/paper/2408.10608/) (2024)
- [Is Meta-Learning Out? Rethinking Unsupervised Few-Shot Classification with Limited Entropy](https://mlsanity.com/paper/2509.13185/) (2025)
- [Towards Enabling Meta-Learning from Target Models](https://mlsanity.com/paper/2104.03736/) (2021)
- [Directional Neural Collapse Explains Few-Shot Transfer in Self-Supervised Learning](https://mlsanity.com/paper/2603.03530/) (2026)
- [Meta-free few-shot learning via representation learning with weight averaging](https://mlsanity.com/paper/2204.12466/) (2022)

## Abstract (arXiv)

Meta-learning algorithms produce feature extractors which achieve state-of-the-art performance on few-shot classification. While the literature is rich with meta-learning methods, little is known about why the resulting feature extractors perform so well. We develop a better understanding of the underlying mechanics of meta-learning and the difference between models trained using meta-learning and models which are trained classically. In doing so, we introduce and verify several hypotheses for why meta-learned models perform better. Furthermore, we develop a regularizer which boosts the performance of standard training routines for few-shot classification. In many cases, our routine outperforms meta-learning while simultaneously running an order of magnitude faster.