# How to Train Your MAML to Excel in Few-Shot Classification

arXiv: https://arxiv.org/abs/2106.16245 · HTML: https://mlsanity.com/paper/2106.16245/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 87 of its 141 references that are in the map.
- First posted on arXiv 30 Jun 2021.

## Builds on (most cited first, 30 of 87)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Meta-Learning with Implicit Gradients](https://mlsanity.com/paper/1909.04630/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)

## Built on by (most cited first, 22 of 22)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks](https://mlsanity.com/paper/2203.17030/) (2022)
- [Property-Aware Relation Networks for Few-Shot Molecular Property Prediction](https://mlsanity.com/paper/2107.07994/) (2021)
- [HyperShot: Few-Shot Learning by Kernel HyperNetworks](https://mlsanity.com/paper/2203.11378/) (2022)
- [Channel Importance Matters in Few-Shot Image Classification](https://mlsanity.com/paper/2206.08126/) (2022)
- [HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworks](https://mlsanity.com/paper/2205.15745/) (2022)
- [Few-Shot Learning with a Strong Teacher](https://mlsanity.com/paper/2107.00197/) (2021)
- [Bi-directional Feature Reconstruction Network for Fine-Grained Few-Shot Image Classification](https://mlsanity.com/paper/2211.17161/) (2022)
- [RankDNN: Learning to Rank for Few-shot Learning](https://mlsanity.com/paper/2211.15320/) (2022)
- ["Define Your Terms" : Enhancing Efficient Offensive Speech Classification with Definition](https://mlsanity.com/paper/2402.03221/) (2024)
- [StratDef: Strategic Defense Against Adversarial Attacks in ML-based Malware Detection](https://mlsanity.com/paper/2202.07568/) (2022)
- [Meta-SysId: A Meta-Learning Approach for Simultaneous Identification and Prediction](https://mlsanity.com/paper/2206.00694/) (2022)
- [FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?](https://mlsanity.com/paper/2307.04114/) (2023)
- [Bridging the Reality Gap of Reinforcement Learning based Traffic Signal Control using Domain Randomization and Meta Learning](https://mlsanity.com/paper/2307.11357/) (2023)
- [Cross-System Categorization of Abnormal Traces in Microservice-Based Systems via Meta-Learning](https://mlsanity.com/paper/2403.18998/) (2024)
- [Rethinking Task Sampling for Few-shot Vision-Language Transfer Learning](https://mlsanity.com/paper/2203.04904/) (2022)
- [Elastically-Constrained Meta-Learner for Federated Learning](https://mlsanity.com/paper/2306.16703/) (2023)
- [Meta-Learning Neural Procedural Biases](https://mlsanity.com/paper/2406.07983/) (2024)
- [Benchmarking for Practice: Few-Shot Time-Series Crop-Type Classification on the EuroCropsML Dataset](https://mlsanity.com/paper/2504.11022/) (2025)
- [Hypernetwork approach to Bayesian MAML](https://mlsanity.com/paper/2210.02796/) (2022)
- [Towards Few-Annotation Learning in Computer Vision: Application to Image Classification and Object Detection tasks](https://mlsanity.com/paper/2311.04888/) (2023)
- [Learn To Learn More Precisely](https://mlsanity.com/paper/2408.04590/) (2024)

## Abstract (arXiv)

Model-agnostic meta-learning (MAML) is arguably one of the most popular meta-learning algorithms nowadays. Nevertheless, its performance on few-shot classification is far behind many recent algorithms dedicated to the problem. In this paper, we point out several key facets of how to train MAML to excel in few-shot classification. First, we find that MAML needs a large number of gradient steps in its inner loop update, which contradicts its common usage in few-shot classification. Second, we find that MAML is sensitive to the class label assignments during meta-testing. Concretely, MAML meta-trains the initialization of an N-way classifier. These N ways, during meta-testing, then have "N!" different permutations to be paired with a few-shot task of N novel classes. We find that these permutations lead to a huge variance of accuracy, making MAML unstable in few-shot classification. Third, we investigate several approaches to make MAML permutation-invariant, among which meta-training a single vector to initialize all the N weight vectors in the classification head performs the best. On benchmark datasets like MiniImageNet and TieredImageNet, our approach, which we name UNICORN-MAML, performs on a par with or even outperforms many recent few-shot classification algorithms, without sacrificing MAML's simplicity.