mlsanityData to 7 Oct 2026

How to Train Your MAML to Excel in Few-Shot Classification

arXiv 2106.16245 ↗First posted 30 Jun 2021cs.LG cs.AI cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

Model-agnostic meta-learning (MAML) is arguably one of the most popular meta-learning algorithms nowadays. Nevertheless, its performance on few-shot classification is far behind many recent algorithms dedicated to the problem. In this paper, we point out several key facets of how to train MAML to excel in few-shot classification. First, we find that MAML needs a large number of gradient steps in its inner loop update, which contradicts its common usage in few-shot classification. Second, we find that MAML is sensitive to the class label assignments during meta-testing. Concretely, MAML meta-trains the initialization of an N-way classifier. These N ways, during meta-testing, then have "N!" different permutations to be paired with a few-shot task of N novel classes. We find that these permutations lead to a huge variance of accuracy, making MAML unstable in few-shot classification. Third, we investigate several approaches to make MAML permutation-invariant, among which meta-training a single vector to initialize all the N weight vectors in the classification head performs the best. On benchmark datasets like MiniImageNet and TieredImageNet, our approach, which we name UNICORN-MAML, performs on a par with or even outperforms many recent few-shot classification algorithms, without sacrificing MAML's simplicity.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 87 of its 141 references that are in the map. First posted on arXiv 30 Jun 2021.

Builds on

87 of 141 references.

References · 87

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  11. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  12. 2020Meta-Learning in Neural Networks: A Surveycited by 944
38 more
  1. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  2. 2019A Closer Look at Few-shot Classificationcited by 679
  3. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  4. 2016MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognitioncited by 621
  5. 2016Learning to learn by gradient descent by gradient descentcited by 616
  6. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  7. 2019Meta-Learning with Implicit Gradientscited by 523
  8. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  9. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  10. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  11. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  12. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  13. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  14. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  15. 2018Meta-learning with differentiable closed-form solverscited by 395
  16. 2016Learning to reinforcement learncited by 369
  17. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  18. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  19. 2018How to train your MAMLcited by 306
  20. 2019A Baseline for Few-Shot Image Classificationcited by 298
  21. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 293
  22. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  23. 2018DropBlock: A regularization method for convolutional networkscited by 261
  24. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  25. 2018Meta-Learning: A Surveycited by 254
  26. 2015YFCC100M: The New Data in Multimedia Researchcited by 234
  27. 2017One-Shot Imitation Learningcited by 232
  28. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  29. 2016Learning to Optimizecited by 195
  30. 2015The Benefit of Multitask Representation Learningcited by 194
  31. 2019SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learningcited by 194
  32. 2020Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformationcited by 183
  33. 2018Low-Shot Learning from Imaginary Datacited by 181
  34. 2015Learning Visual Features from Large Weakly Supervised Datacited by 165
  35. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  36. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  37. 2019Feature-Critic Networks for Heterogeneous Domain Generalizationcited by 130
  38. 2020Negative Margin Matters: Understanding Margin in Few-shot Classificationcited by 129

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 11; 2023: 4; 2024: 4; 2025: 1.

22 papers.

Sort:

By area:

  1. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  2. 2022Few-Shot Class-Incremental Learning by Sampling Multi-Phase Taskscited by 87
  3. 2021Property-Aware Relation Networks for Few-Shot Molecular Property Predictioncited by 41
  4. 2022HyperShot: Few-Shot Learning by Kernel HyperNetworkscited by 24
  5. 2022Channel Importance Matters in Few-Shot Image Classificationcited by 20
  6. 2021Few-Shot Learning with a Strong Teachercited by 16
  7. 2022HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworkscited by 16
  8. 2022Bi-directional Feature Reconstruction Network for Fine-Grained Few-Shot Image Classificationcited by 12
  9. 2022RankDNN: Learning to Rank for Few-shot Learningcited by 7
  10. 2024"Define Your Terms" : Enhancing Efficient Offensive Speech Classification with Definitioncited by 7
  11. 2022StratDef: Strategic Defense Against Adversarial Attacks in ML-based Malware Detectioncited by 3
  12. 2022Meta-SysId: A Meta-Learning Approach for Simultaneous Identification and Predictioncited by 3
10 more
  1. 2023FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?cited by 2
  2. 2023Bridging the Reality Gap of Reinforcement Learning based Traffic Signal Control using Domain Randomization and Meta Learningcited by 2
  3. 2024Cross-System Categorization of Abnormal Traces in Microservice-Based Systems via Meta-Learningcited by 2
  4. 2022Rethinking Task Sampling for Few-shot Vision-Language Transfer Learningcited by 1
  5. 2023Elastically-Constrained Meta-Learner for Federated Learningcited by 1
  6. 2024Meta-Learning Neural Procedural Biasescited by 1
  7. 2025Benchmarking for Practice: Few-Shot Time-Series Crop-Type Classification on the EuroCropsML Datasetcited by 1
  8. 2022Hypernetwork approach to Bayesian MAMLcited by 0
  9. 2023Towards Few-Annotation Learning in Computer Vision: Application to Image Classification and Object Detection taskscited by 0
  10. 2024Learn To Learn More Preciselycited by 0