mlsanityData to 14 Oct 2026

Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile

arXiv 2206.01944 ↗First posted 4 Jun 2022cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent years have seen a surge of interest in meta-learning techniques for tackling the few-shot learning (FSL) problem. However, the meta-learner is prone to overfitting since there are only a few available samples, which can be identified as sampling noise on a clean dataset. Moreover, when handling the data with noisy labels, the meta-learner could be extremely sensitive to label noise on a corrupted dataset. To address these two challenges, we present Eigen-Reptile (ER) that updates the meta-parameters with the main direction of historical task-specific parameters to alleviate sampling and label noise. Specifically, the main direction is computed in a fast way, where the scale of the calculated matrix is related to the number of gradient steps instead of the number of parameters. Furthermore, to obtain a more accurate main direction for Eigen-Reptile in the presence of many noisy labels, we further propose Introspective Self-paced Learning (ISPL). We have theoretically and experimentally demonstrated the soundness and effectiveness of the proposed Eigen-Reptile and ISPL. Particularly, our experiments on different tasks show that the proposed method is able to outperform or achieve highly competitive performance compared with other gradient-based methods with or without noisy labels. The code and data for the proposed method are provided for research purposes https://github.com/Anfeather/Eigen-Reptile.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 43 of its 62 references that are in the map. First posted on arXiv 4 Jun 2022.

Builds on

43 of 62 references.

References · 43

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  4. 2016Matching Networks for One Shot Learningcited by 2,255
  5. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  6. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  7. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  8. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  9. 2016Learning to learn by gradient descent by gradient descentcited by 620
  10. 2019Meta-Learning with Implicit Gradientscited by 525
  11. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  12. 2018Meta-learning with differentiable closed-form solverscited by 397
31 more
  1. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  2. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  3. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 379
  4. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 328
  5. 2017Three Factors Influencing Minima in SGDcited by 273
  6. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 243
  7. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  8. 2021Free Lunch for Few-shot Learning: Distribution Calibrationcited by 165
  9. 2019Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networkscited by 159
  10. 2018Task-Agnostic Meta-Learning for Few-shot Learningcited by 151
  11. 2015Adding Gradient Noise Improves Learning for Very Deep Networkscited by 145
  12. 2019Meta-Learning with Warped Gradient Descentcited by 98
  13. 2018Masking: A New Perspective of Noisy Supervisioncited by 93
  14. 2019Meta-Learning without Memorizationcited by 92
  15. 2020DPGN: Distribution Propagation Graph Network for Few-shot Learningcited by 79
  16. 2019Meta-Curvaturecited by 70
  17. 2016A Variational Analysis of Stochastic Gradient Algorithmscited by 57
  18. 2019Few-Shot Learning with Embedded Class Models and Shot-Free Meta Trainingcited by 57
  19. 2020Meta-Learning Requires Meta-Augmentationcited by 57
  20. 2021Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learningcited by 49
  21. 2020Empirical Bayes Transductive Meta-Learning with Synthetic Gradientscited by 48
  22. 2017Deep Learning from Noisy Image Labels with Quality Embeddingcited by 46
  23. 2019Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigationcited by 40
  24. 2019A Theoretical Analysis of the Number of Shots in Few-Shot Learningcited by 39
  25. 2020Data Augmentation for Meta-Learningcited by 38
  26. 2021Learning where to learn: Gradient sparsity in meta and continual learningcited by 33
  27. 2021Meta Discovery: Learning to Discover Novel Classes given Very Limited Datacited by 31
  28. 2020Mixture-based Feature Space Learning for Few-shot Image Classificationcited by 23
  29. 2018Concept Learning with Energy-Based Modelscited by 14
  30. 2021Learning Diverse-Structured Networks for Adversarial Robustnesscited by 12
  31. 2018Fast Context Adaptation via Meta-Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3; 2025: 1.

6 papers.

  1. 2024Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitationcited by 18
  2. 2023Improving Vision Anomaly Detection with the Guidance of Language Modalitycited by 7
  3. 2024Towards Few-Shot Learning in the Open World: A Review and Beyondcited by 5
  4. 2024Meta-Learning with Heterogeneous Taskscited by 2
  5. 2023BatMan-CLR: Making Few-shots Meta-Learners Resilient Against Label Noisecited by 0
  6. 2025FAF: A Feature-Adaptive Framework for Few-Shot Time Series Forecastingcited by 0