Data to 7 Oct 2026

Learning to Reweight Examples for Robust Deep Learning

arXiv 1803.09050 ↗First posted 24 Mar 2018cs.LG stat.MLcited by 670

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns. However, they can also easily overfit to training set biases and label noises. In addition to various regularizers, example reweighting algorithms are popular solutions to these problems, but they require careful tuning of additional hyperparameters, such as example mining schedules and regularization hyperparameters. In contrast to past reweighting methods, which typically consist of functions of the cost value of each example, in this work we propose a novel meta-learning algorithm that learns to assign weights to training examples based on their gradient directions. To determine the example weights, our method performs a meta gradient descent step on the current mini-batch example weights (which are initialized from zero) to minimize the loss on a clean unbiased validation set. Our proposed method can be easily implemented on any type of deep network, does not require any additional hyperparameter tuning, and achieves impressive performance on class imbalance and corrupted label problems where only a small amount of clean validation data is available.

Key facts. Cited by 670 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 172; 2022: 135; 2023: 115; 2024: 114; 2025: 65; 2026: 69.

670 papers.

Sort:

By area:

2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
2021Gradient Matching for Domain Generalizationcited by 258
2021Curriculum Learning: A Surveycited by 257
2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 233
2021Parametric Contrastive Learningcited by 217
2021Distribution Alignment: A Unified Framework for Long-tail Visual Recognitioncited by 198
2021CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learningcited by 174
2021Contrastive Learning based Hybrid Networks for Long-Tailed Image Classificationcited by 130
2021Self-supervised Learning is More Robust to Dataset Imbalancecited by 122
2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
2022Long-Tailed Recognition via Weight Balancingcited by 107

38 more

2021Regularizing Generative Adversarial Networks under Limited Datacited by 105
2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
2021MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognitioncited by 99
2021Provably End-to-end Label-Noise Learning without Anchor Pointscited by 97
2021ResLT: Residual Learning for Long-tailed Recognitioncited by 96
2026TIP: Token Importance in On-Policy Distillationcited by 90
2022Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimationcited by 88
2021FINE Samples for Learning with Noisy Labelscited by 82
2021Jo-SRC: A Contrastive Approach for Combating Noisy Labelscited by 80
2021Robust Audio-Visual Instance Discriminationcited by 80
2021OOD-GNN: Out-of-Distribution Generalized Graph Neural Networkcited by 78
2023LLMaAA: Making Large Language Models as Active Annotatorscited by 78
2022A Survey on Long-Tailed Visual Recognitioncited by 75
2023Long-tailed Visual Recognition via Gaussian Clouded Logit Adjustmentcited by 75
2021RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learningcited by 72
2023DELTA: degradation-free fully test-time adaptationcited by 72
2021Learning Fast Sample Re-weighting Without Reward Datacited by 69
2022Learn From All: Erasing Attention Consistency for Noisy Label Facial Expression Recognitioncited by 67
2021ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learningcited by 65
2022Nested Collaborative Learning for Long-Tailed Visual Recognitioncited by 63
2023Guiding Pseudo-labels with Uncertainty Estimation for Source-free Unsupervised Domain Adaptationcited by 63
2022The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph Generationcited by 60
2022Part-based Pseudo Label Refinement for Unsupervised Person Re-identificationcited by 59
2022Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluationcited by 59
2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyondcited by 59
2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 58
2021NGC: A Unified Framework for Learning with Open-World Noisy Datacited by 58
2022Confidence Score for Source-Free Unsupervised Domain Adaptationcited by 58
2021Topology-Imbalance Learning for Semi-Supervised Node Classificationcited by 56
2021Exploring Memorization in Adversarial Trainingcited by 55
2022DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotationscited by 54
2022Meta Balanced Network for Fair Face Recognitioncited by 53
2021DASO: Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learningcited by 51
2021Adversarial Robustness under Long-Tailed Distributioncited by 50
2021Self-Training with Weak Supervisioncited by 49
2022Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learningcited by 49
2021Sample Selection for Fair and Robust Trainingcited by 48
2022Investigating Why Contrastive Learning Benefits Robustness Against Label Noisecited by 46

50 most cited of 670 All 670 (JSON)