mlsanityData to 14 Oct 2026

On Hard Episodes in Meta-Learning

arXiv 2110.11190 ↗First posted 21 Oct 2021cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing meta-learners primarily focus on improving the average task accuracy across multiple episodes. Different episodes, however, may vary in hardness and quality leading to a wide gap in the meta-learner's performance across episodes. Understanding this issue is particularly critical in industrial few-shot settings, where there is limited control over test episodes as they are typically uploaded by end-users. In this paper, we empirically analyse the behaviour of meta-learners on episodes of varying hardness across three standard benchmark datasets: CIFAR-FS, mini-ImageNet, and tiered-ImageNet. Surprisingly, we observe a wide gap in accuracy of around 50% between the hardest and easiest episodes across all the standard benchmarks and meta-learners. We additionally investigate various properties of hard episodes and highlight their connection to catastrophic forgetting during meta-training. To address the issue of sub-par performance on hard episodes, we investigate and benchmark different meta-training strategies based on adversarial training and curriculum learning. We find that adversarial training strategies are much more powerful than curriculum learning in improving the prediction performance on hard episodes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 21 Oct 2021.

Builds on

22 of 30 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  5. 2016Matching Networks for One Shot Learningcited by 2,255
  6. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
  7. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  8. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
  9. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  10. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  11. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  12. 2018Meta-learning with differentiable closed-form solverscited by 397
10 more
  1. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  2. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  3. 2019A Baseline for Few-Shot Image Classificationcited by 298
  4. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
  5. 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
  6. 2017Few-Shot Image Recognition by Predicting Parameters from Activationscited by 122
  7. 2020Self-training for Few-shot Transfer Across Extreme Task Differencescited by 76
  8. 2020Data Augmentation for Meta-Learningcited by 38
  9. 2020MaxUp: A Simple Way to Improve Generalization of Neural Network Trainingcited by 27
  10. 2021Uniform Sampling over Episode Difficultycited by 6

Built on by

Not cited yet.