mlsanityData to 14 Oct 2026

Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization

arXiv 2006.15766 ↗First posted 29 Jun 2020cs.LG stat.MLcited by 49

Not on the map. Why?

Abstract (arXiv)

Real-world large-scale datasets are heteroskedastic and imbalanced -- labels have varying levels of uncertainty and label distributions are long-tailed. Heteroskedasticity and imbalance challenge deep learning algorithms due to the difficulty of distinguishing among mislabeled, ambiguous, and rare examples. Addressing heteroskedasticity and imbalance simultaneously is under-explored. We propose a data-dependent regularization technique for heteroskedastic datasets that regularizes different regions of the input space differently. Inspired by the theoretical derivation of the optimal regularization strength in a one-dimensional nonparametric classification setting, our approach adaptively regularizes the data points in higher-uncertainty, lower-density regions more heavily. We test our method on several benchmark tasks, including a real-world heteroskedastic and imbalanced dataset, WebVision. Our experiments corroborate our theory and demonstrate a significant improvement over other methods in noise-robust deep learning.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 16; 2023: 9; 2024: 6; 2025: 3; 2026: 2.

49 papers.

Sort:

By area:

  1. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 523
  2. 2021Deep Long-Tailed Learning: A Surveycited by 417
  3. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 381
  4. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  5. 2021Open-World Semi-Supervised Learningcited by 165
  6. 2021Self-supervised Learning is More Robust to Dataset Imbalancecited by 123
  7. 2021FINE Samples for Learning with Noisy Labelscited by 82
  8. 2022Investigating Why Contrastive Learning Benefits Robustness Against Label Noisecited by 46
  9. 2021On Tilted Losses in Machine Learning: Theory and Applicationscited by 43
  10. 2023Robust Learning with Progressive Data Expansion Against Spurious Correlationcited by 39
  11. 2021Correlated Input-Dependent Label Noise in Large-Scale Image Classificationcited by 37
  12. 2021Robust Long-Tailed Learning under Label Noisecited by 36
37 more
  1. 2022SoLar: Sinkhorn Label Refinery for Imbalanced Partial-Label Learningcited by 31
  2. 2022CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learningcited by 24
  3. 2022When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration Methodcited by 21
  4. 2021Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrectioncited by 20
  5. 2022Identifying Hard Noise in Long-Tailed Sample Distributioncited by 19
  6. 2022Label-Noise Learning with Intrinsically Long-Tailed Datacited by 17
  7. 2023Late Stopping: Avoiding Confidently Learning from Mislabeled Examplescited by 17
  8. 2021Delving into Sample Loss Curve to Embrace Noisy and Imbalanced Datacited by 16
  9. 2023Fairness through Aleatoric Uncertaintycited by 16
  10. 2022Offline RL With Realistic Datasets: Heteroskedasticity and Support Constraintscited by 13
  11. 2023Wrapped Cauchy Distributed Angular Softmax for Long-Tailed Visual Recognitioncited by 11
  12. 2024A Survey of Deep Long-Tail Classification Advancementscited by 11
  13. 2021Prototypical Classifier for Robust Class-Imbalanced Learningcited by 10
  14. 2023Fine tuning Pre trained Models for Robustness Under Noisy Labelscited by 8
  15. 2022Dynamic Loss For Robust Learningcited by 6
  16. 2023Bias Amplification Enhances Minority Group Performancecited by 6
  17. 2022Neural Active Learning on Heteroskedastic Distributionscited by 5
  18. 2022Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithmscited by 4
  19. 2022A Gift from Label Smoothing: Robust Training with Adaptive Label Smoothing via Auxiliary Classifier under Label Noisecited by 4
  20. 2021Batch Inverse-Variance Weighting: Deep Heteroscedastic Regressioncited by 3
  21. 2023REDUCR: Robust Data Downsampling Using Class Priority Reweightingcited by 3
  22. 2024Extracting Clean and Balanced Subset for Noisy Long-tailed Classificationcited by 3
  23. 2022Learning from Long-Tailed Noisy Data with Sample Selection and Balanced Losscited by 2
  24. 2023Pushing the Accuracy-Group Robustness Frontier with Introspective Self-playcited by 2
  25. 2024Distribution-Aware Robust Learning from Long-Tailed Data with Noisy Labelscited by 2
  26. 2025Classifying Long-tailed and Label-noise Data via Disentangling and Unlearningcited by 2
  27. 2022Maximising the Utility of Validation Sets for Imbalanced Noisy-label Meta-learningcited by 1
  28. 2022Denoising after Entropy-based Debiasing A Robust Training Method for Dataset Bias with Noisy Labelscited by 1
  29. 2023Data-Centric Long-Tailed Image Recognitioncited by 1
  30. 2024A Bilevel Optimization Framework for Imbalanced Data Classificationcited by 1
  31. 2025Dual-granularity Sinkhorn Distillation for Enhanced Learning from Long-tailed Noisy Datacited by 1
  32. 2026CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labelscited by 1
  33. 2021Solution for Large-scale Long-tailed Recognition with Noisy Labelscited by 0
  34. 2024Addressing Long-Tail Noisy Label Learning Problems: a Two-Stage Solution with Label Refurbishment Considering Label Raritycited by 0
  35. 2024Jump-teaching: Combating Sample Selection Bias via Temporal Disagreementcited by 0
  36. 2025Process Optimization and Deployment for Sensor-Based Human Activity Recognition Based on Deep Learningcited by 0
  37. 2026Learning from Imperfect Text Guidance: Robust Long-Tail Visual Recognition with High-Noise Labelcited by 0