mlsanityData to 7 Oct 2026

Perturbing the Gradient for Alleviating Meta Overfitting

arXiv 2405.12299 ↗First posted 20 May 2024cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The reason for Meta Overfitting can be attributed to two factors: Mutual Non-exclusivity and the Lack of diversity, consequent to which a single global function can fit the support set data of all the meta-training tasks and fail to generalize to new unseen tasks. This issue is evidenced by low error rates on the meta-training tasks, but high error rates on new tasks. However, there can be a number of novel solutions to this problem keeping in mind any of the two objectives to be attained, i.e. to increase diversity in the tasks and to reduce the confidence of the model for some of the tasks. In light of the above, this paper proposes a number of solutions to tackle meta-overfitting on few-shot learning settings, such as few-shot sinusoid regression and few shot classification. Our proposed approaches demonstrate improved generalization performance compared to state-of-the-art baselines for learning in a non-mutually exclusive task setting. Overall, this paper aims to provide insights into tackling overfitting in meta-learning and to advance the field towards more robust and generalizable models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 20 May 2024.

Builds on

25 of 33 references.

References · 25

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  3. 2016Matching Networks for One Shot Learningcited by 2,246
  4. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  5. 2019Meta-Learning with Implicit Gradientscited by 523
  6. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  7. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  8. 2018Conditional Neural Processescited by 421
  9. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  10. 2017Meta Networkscited by 268
  11. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  12. 2019Online Meta-Learningcited by 137
13 more
  1. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  2. 2020A Survey of Deep Meta-Learningcited by 99
  3. 2019Meta-Learning without Memorizationcited by 92
  4. 2019ROBEL: Robotics Benchmarks for Learning with Low-Cost Robotscited by 70
  5. 2018Diverse Few-Shot Text Classification with Multiple Metricscited by 61
  6. 2021BERT Learns to Teach: Knowledge Distillation with Meta Learningcited by 61
  7. 2020Meta-Learning Requires Meta-Augmentationcited by 57
  8. 2019The Omniglot challenge: a 3-year progress reportcited by 54
  9. 2020Rapidly Adaptable Legged Robots via Evolutionary Meta-Learningcited by 42
  10. 2019Multi-Level Matching and Aggregation Network for Few-Shot Relation Classificationcited by 32
  11. 2022Understanding and Combating Robust Overfitting via Input Loss Landscape Analysis and Regularizationcited by 22
  12. 2020Meta-Learning with Network Pruningcited by 14
  13. 2019Meta-Learning Initializations for Image Segmentationcited by 12

Built on by

Not cited yet.