mlsanityData to 7 Oct 2026

Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithms

arXiv 2206.05749 ↗First posted 12 Jun 2022cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Due to the poor generalization performance of traditional empirical risk minimization (ERM) in the case of distributional shift, Out-of-Distribution (OoD) generalization algorithms receive increasing attention. However, OoD generalization algorithms overlook the great variance in the quality of training data, which significantly compromises the accuracy of these methods. In this paper, we theoretically reveal the relationship between training data quality and algorithm performance and analyze the optimal regularization scheme for Lipschitz regularized invariant risk minimization. A novel algorithm is proposed based on the theoretical results to alleviate the influence of low-quality data at both the sample level and the domain level. The experiments on both the regression and classification benchmarks validate the effectiveness of our method with statistical significance.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 12 Jun 2022.

Builds on

38 of 56 references.

References · 38

  1. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  4. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  5. 2020In Search of Lost Domain Generalizationcited by 1,042
  6. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  7. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  8. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  9. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  10. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  11. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  12. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
26 more
  1. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
  2. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  3. 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
  4. 2020Improve Unsupervised Domain Adaptation with Mixup Trainingcited by 185
  5. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  6. 2018Learning to Learn from Noisy Labeled Datacited by 160
  7. 2020Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learningcited by 142
  8. 2020Learning explanations that are hard to varycited by 132
  9. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  10. 2020Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspectivecited by 120
  11. 2018Gradient Harmonized Single-stage Detectorcited by 114
  12. 2019Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentationcited by 71
  13. 2019Stable Learning via Sample Reweightingcited by 70
  14. 2020Empirical or Invariant Risk Minimization? A Sample Complexity Perspectivecited by 62
  15. 2021Out of Distribution Generalization in Machine Learningcited by 59
  16. 2019Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margincited by 56
  17. 2018Stable Prediction across Unknown Environmentscited by 55
  18. 2018Algorithms and Theory for Multiple-Source Adaptationcited by 53
  19. 2020Posterior Re-calibration for Imbalanced Datasetscited by 53
  20. 2020When is invariance useful in an Out-of-Distribution Generalization problem ?cited by 52
  21. 2020Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularizationcited by 49
  22. 2019Reducing Domain Gap by Reducing Style Biascited by 35
  23. 2020MESA: Boost Ensemble Imbalanced Learning with MEta-SAmplercited by 25
  24. 2021On the benefits of representation regularization in invariance based domain generalizationcited by 17
  25. 2018Metric-Optimized Example Weightscited by 8
  26. 2013Smoothing splines with varying smoothing parametercited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2.

4 papers.

  1. 2023Unlocking the Power of Open Set : A New Perspective for Open-Set Noisy Label Learningcited by 8
  2. 2023Learning Optimal Features via Partial Invariancecited by 5
  3. 2024Data Quality in Edge Machine Learning: A State-of-the-Art Surveycited by 0
  4. 2024Invariant Correlation of Representation with Label: Enhancing Domain Generalization in Noisy Environmentscited by 0