# Regretful Decisions under Label Noise

arXiv: https://arxiv.org/abs/2504.09330 · HTML: https://mlsanity.com/paper/2504.09330/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 59 references that are in the map.
- First posted on arXiv 12 Apr 2025.

## Builds on (most cited first, 27 of 27)

- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates](https://mlsanity.com/paper/1910.03231/) (2019)
- [Machine Learning with a Reject Option: A survey](https://mlsanity.com/paper/2107.11277/) (2021)
- [Provably End-to-end Label-Noise Learning without Anchor Points](https://mlsanity.com/paper/2102.02400/) (2021)
- [To Smooth or Not? When Label Smoothing Meets Noisy Labels](https://mlsanity.com/paper/2106.04149/) (2021)
- [Characterizing Fairness Over the Set of Good Models Under Selective Labels](https://mlsanity.com/paper/2101.00352/) (2021)
- [Optimal strategies for reject option classifiers](https://mlsanity.com/paper/2101.12523/) (2021)
- [Predictive Multiplicity in Probabilistic Classification](https://mlsanity.com/paper/2206.01131/) (2022)
- [Understanding Instance-Level Label Noise: Disparate Impacts and Treatments](https://mlsanity.com/paper/2102.05336/) (2021)
- [Rank List Sensitivity of Recommender Systems to Interaction Perturbations](https://mlsanity.com/paper/2201.12686/) (2022)
- [Arbitrary Decisions are a Hidden Cost of Differentially Private Training](https://mlsanity.com/paper/2302.14517/) (2023)
- [The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictions](https://mlsanity.com/paper/2304.10655/) (2023)
- [Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Space](https://mlsanity.com/paper/2206.01106/) (2022)
- [Predictive Churn with the Set of Good Models](https://mlsanity.com/paper/2402.07745/) (2024)
- [Deep learning with noisy labels in medical prediction problems: a scoping review](https://mlsanity.com/paper/2403.13111/) (2024)
- [Feature Responsiveness Scores: Model-Agnostic Explanations for Recourse](https://mlsanity.com/paper/2410.22598/) (2024)
- [When Personalization Harms: Reconsidering the Use of Group Attributes in Prediction](https://mlsanity.com/paper/2206.02058/) (2022)
- [Participatory Personalization in Classification](https://mlsanity.com/paper/2302.03874/) (2023)
- [Prediction without Preclusion: Recourse Verification with Reachable Sets](https://mlsanity.com/paper/2308.12820/) (2023)

## Built on by (most cited first, 1 of 1)

- [Observational Multiplicity](https://mlsanity.com/paper/2507.23136/) (2025)

## Abstract (arXiv)

Machine learning models are routinely used to support decisions that affect individuals -- be it to screen a patient for a serious illness or to gauge their response to treatment. In these tasks, we are limited to learning models from datasets with noisy labels. In this paper, we study the instance-level impact of learning under label noise. We introduce a notion of regret for this regime, which measures the number of unforeseen mistakes due to noisy labels. We show that standard approaches to learning under label noise can return models that perform well at a population-level while subjecting individuals to a lottery of mistakes. We present a versatile approach to estimate the likelihood of mistakes at the individual-level from a noisy dataset by training models over plausible realizations of datasets without label noise. This is supported by a comprehensive empirical study of label noise in clinical prediction tasks. Our results reveal how failure to anticipate mistakes can compromise model reliability and adoption -- we demonstrate how we can address these challenges by anticipating and avoiding regretful decisions.