mlsanityData to 14 Oct 2026

Maximising the Utility of Validation Sets for Imbalanced Noisy-label Meta-learning

arXiv 2208.08132 ↗First posted 17 Aug 2022cs.LG cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Meta-learning is an effective method to handle imbalanced and noisy-label learning, but it depends on a validation set containing randomly selected, manually labelled and balanced distributed samples. The random selection and manual labelling and balancing of this validation set is not only sub-optimal for meta-learning, but it also scales poorly with the number of classes. Hence, recent meta-learning papers have proposed ad-hoc heuristics to automatically build and label this validation set, but these heuristics are still sub-optimal for meta-learning. In this paper, we analyse the meta-learning algorithm and propose new criteria to characterise the utility of the validation set, based on: 1) the informativeness of the validation set; 2) the class distribution balance of the set; and 3) the correctness of the labels of the set. Furthermore, we propose a new imbalanced noisy-label meta-learning (INOLML) algorithm that automatically builds a validation set by maximising its utility using the criteria above. Our method shows significant improvements over previous meta-learning approaches and sets the new state-of-the-art on several benchmarks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 69 of its 96 references that are in the map. First posted on arXiv 17 Aug 2022.

Builds on

69 of 96 references.

References · 69

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  5. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  6. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  7. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  8. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  9. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
  10. 2018Learning to Reweight Examples for Robust Deep Learningcited by 675
  11. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  12. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
38 more
  1. 2020Long-tail learning via logit adjustmentcited by 618
  2. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 554
  3. 2015Towards Open Set Deep Networkscited by 538
  4. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 500
  5. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
  6. 2021Deep Long-Tailed Learning: A Surveycited by 417
  7. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 416
  8. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  9. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  10. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 328
  11. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  12. 2017Decoupling "when to update" from "how to update"cited by 230
  13. 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
  14. 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
  15. 2019CenterMask : Real-Time Anchor-Free Instance Segmentationcited by 201
  16. 2018Dimensionality-Driven Learning with Noisy Labelscited by 173
  17. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  18. 2018Learning to Learn from Noisy Labeled Datacited by 160
  19. 2019NLNL: Negative Learning for Noisy Labelscited by 151
  20. 2020Remix: Rebalanced Mixupcited by 144
  21. 2020Feature Space Augmentation for Long-Tailed Datacited by 136
  22. 2020Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspectivecited by 120
  23. 2016Range Loss for Deep Face Recognition with Long-tailcited by 111
  24. 2018Iterative Learning with Open-set Noisy Labelscited by 108
  25. 2020The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentationcited by 108
  26. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  27. 2017Learning with Biased Complementary Labelscited by 93
  28. 2020Multi-Objective Interpolation Training for Robustness to Label Noisecited by 86
  29. 2020Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noisecited by 82
  30. 2022UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learningcited by 77
  31. 2021FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentationcited by 76
  32. 2021Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labelscited by 74
  33. 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
  34. 2021Learning Fast Sample Re-weighting Without Reward Datacited by 69
  35. 2019Robust Inference via Generative Classifiers for Handling Noisy Labelscited by 68
  36. 2022Learning with Neighbor Consistency for Noisy Labelscited by 55
  37. 2019On Symmetric Losses for Learning from Corrupted Labelscited by 53
  38. 2021Adversarial Robustness under Long-Tailed Distributioncited by 50

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Dynamic Loss For Robust Learningcited by 6