mlsanityData to 14 Oct 2026

Regretful Decisions under Label Noise

arXiv 2504.09330 ↗First posted 12 Apr 2025cs.LG stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Machine learning models are routinely used to support decisions that affect individuals -- be it to screen a patient for a serious illness or to gauge their response to treatment. In these tasks, we are limited to learning models from datasets with noisy labels. In this paper, we study the instance-level impact of learning under label noise. We introduce a notion of regret for this regime, which measures the number of unforeseen mistakes due to noisy labels. We show that standard approaches to learning under label noise can return models that perform well at a population-level while subjecting individuals to a lottery of mistakes. We present a versatile approach to estimate the likelihood of mistakes at the individual-level from a noisy dataset by training models over plausible realizations of datasets without label noise. This is supported by a comprehensive empirical study of label noise in clinical prediction tasks. Our results reveal how failure to anticipate mistakes can compromise model reliability and adoption -- we demonstrate how we can address these challenges by anticipating and avoiding regretful decisions.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 59 references that are in the map. First posted on arXiv 12 Apr 2025.

Builds on

27 of 59 references.

References · 27

  1. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  2. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  3. 2017Selective Classification for Deep Neural Networkscited by 852
  4. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  5. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  6. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  7. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 285
  8. 2016Model-Agnostic Interpretability of Machine Learningcited by 219
  9. 2019Predictive Multiplicity in Classificationcited by 166
  10. 2019Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Ratescited by 142
  11. 2021Machine Learning with a Reject Option: A surveycited by 109
  12. 2021Provably End-to-end Label-Noise Learning without Anchor Pointscited by 97
15 more
  1. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  2. 2021Characterizing Fairness Over the Set of Good Models Under Selective Labelscited by 65
  3. 2021Optimal strategies for reject option classifierscited by 61
  4. 2022Predictive Multiplicity in Probabilistic Classificationcited by 44
  5. 2021Understanding Instance-Level Label Noise: Disparate Impacts and Treatmentscited by 22
  6. 2022Rank List Sensitivity of Recommender Systems to Interaction Perturbationscited by 20
  7. 2023Arbitrary Decisions are a Hidden Cost of Differentially Private Trainingcited by 19
  8. 2023The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictionscited by 17
  9. 2022Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Spacecited by 15
  10. 2024Predictive Churn with the Set of Good Modelscited by 15
  11. 2024Deep learning with noisy labels in medical prediction problems: a scoping reviewcited by 10
  12. 2024Feature Responsiveness Scores: Model-Agnostic Explanations for Recoursecited by 8
  13. 2022When Personalization Harms: Reconsidering the Use of Group Attributes in Predictioncited by 6
  14. 2023Participatory Personalization in Classificationcited by 5
  15. 2023Prediction without Preclusion: Recourse Verification with Reachable Setscited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Observational Multiplicitycited by 2