mlsanityData to 7 Oct 2026

Smoothly Giving up: Robustness for Simple Models

arXiv 2302.09114 ↗First posted 17 Feb 2023cs.LG cs.IT math.ITcited by 0

Not on the map. Why?

Abstract (arXiv)

There is a growing need for models that are interpretable and have reduced energy and computational cost (e.g., in health care analytics and federated learning). Examples of algorithms to train such models include logistic regression and boosting. However, one challenge facing these algorithms is that they provably suffer from label noise; this has been attributed to the joint interaction between oft-used convex loss functions and simpler hypothesis classes, resulting in too much emphasis being placed on outliers. In this work, we use the margin-based α-loss, which continuously tunes between canonical convex and quasi-convex losses, to robustly train simple models. We show that the α hyperparameter smoothly introduces non-convexity and offers the benefit of "giving up" on noisy training examples. We also provide results on the Long-Servedio dataset for boosting and a COVID-19 survey dataset for logistic regression, highlighting the efficacy of our approach across multiple relevant domains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 80 references that are in the map. First posted on arXiv 17 Feb 2023.

Builds on

30 of 80 references.

References · 30

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2021Convex optimizationcited by 2,479
  4. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  5. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  6. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  7. 2019AutoML: A Survey of the State-of-the-Artcited by 388
  8. 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
  9. 2017A General and Adaptive Robust Loss Functioncited by 236
  10. 2017Deep Learning is Robust to Massive Label Noisecited by 164
  11. 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
  12. 2015Learning with Symmetric Label Noise: The Importance of Being Unhingedcited by 120
18 more
  1. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 112
  2. 2016The Landscape of Empirical Risk for Non-convex Lossescited by 86
  3. 2019Combating Label Noise in Deep Learning Using Abstentioncited by 86
  4. 2022PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functionscited by 62
  5. 2021Instance-dependent Label-noise Learning under a Structural Causal Modelcited by 50
  6. 2020Regularization via Structural Label Smoothingcited by 29
  7. 2021Accuracy, Interpretability, and Differential Privacy via Explainable Boostingcited by 24
  8. 2019A Tunable Loss Function for Robust Classification: Calibration, Landscape, and Generalizationcited by 23
  9. 2019A Tunable Loss Function for Binary Classificationcited by 11
  10. 2021Being Properly Impropercited by 11
  11. 2021Realizing GANs via a Tunable Loss Functioncited by 10
  12. 2020Learning Not to Learn in the Presence of Noisy Labelscited by 9
  13. 2013Boosting with the Logistic Loss is Consistentcited by 3
  14. 2020On the alpha-loss Landscape in the Logistic Modelcited by 3
  15. 2014Non-Convex Boosting Overcomes Random Label Noisecited by 2
  16. 2021The perils of being unhinged: On the accuracy of classifiers minimizing a noise-robust convex losscited by 2
  17. 2022Agnostic Learnability of Halfspaces via Logistic Losscited by 2
  18. 2022What killed the Convex Booster ?cited by 1

Built on by

Not cited yet.