# Learning with Confidence: Training Better Classifiers from Soft Labels

arXiv: https://arxiv.org/abs/2409.16071 · HTML: https://mlsanity.com/paper/2409.16071/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 45 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 6 of 6)

- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [Label Distribution Learning](https://mlsanity.com/paper/1408.6027/) (2014)
- [Deep Label Distribution Learning with Label Ambiguity](https://mlsanity.com/paper/1611.01731/) (2016)
- [Confidence Scores Make Instance-dependent Label-noise Learning Possible](https://mlsanity.com/paper/2001.03772/) (2020)
- [Knowledge Learning with Crowdsourcing: A Brief Review and Systematic Perspective](https://mlsanity.com/paper/2206.09315/) (2022)
- [Generating the Ground Truth: Synthetic Data for Soft Label and Label Noise Research](https://mlsanity.com/paper/2309.04318/) (2023)

## Built on by (most cited first, 6 of 6)

- [Perspective Transition of Large Language Models for Solving Subjective Tasks](https://mlsanity.com/paper/2501.09265/) (2025)
- [Building Age Estimation: A New Multi-Modal Benchmark Dataset and Community Challenge](https://mlsanity.com/paper/2502.13818/) (2025)
- [Distributions In, Distributions Out: The Case for Soft-Label Training](https://mlsanity.com/paper/2511.14117/) (2025)
- [Quantifying Ambiguity in Categorical Annotations: A Measure and Statistical Inference Framework](https://mlsanity.com/paper/2510.04366/) (2025)
- [Transformer-Based Active Learning for Data-Efficient Vaccine Epitope Selection in PRRS](https://mlsanity.com/paper/2606.28659/) (2026)
- [Learning Emotion from Motion: Kinetic Multi-Stream Skeleton Modeling with Metadata-Conditioned Weak Label Distributions](https://mlsanity.com/paper/2607.17121/) (2026)

## Abstract (arXiv)

In supervised machine learning, models are typically trained using data with hard labels, i.e., definite assignments of class membership. This traditional approach, however, does not take the inherent uncertainty in these labels into account. We investigate whether incorporating label uncertainty, represented as discrete probability distributions over the class labels -- known as soft labels -- improves the predictive performance of classification models. We first demonstrate the potential value of soft label learning (SLL) for estimating model parameters in a simulation experiment, particularly for limited sample sizes and imbalanced data. Subsequently, we compare the performance of various wrapper methods for learning from both hard and soft labels using identical base classifiers. On real-world-inspired synthetic data with clean labels, the SLL methods consistently outperform hard label methods. Since real-world data is often noisy and precise soft labels are challenging to obtain, we study the effect that noisy probability estimates have on model performance. Alongside conventional noise models, our study examines four types of miscalibration that are known to affect human annotators. The results show that SLL methods outperform the hard label methods in the majority of settings. Finally, we evaluate the methods on a real-world dataset with confidence scores, where the SLL methods are shown to match the traditional methods for predicting the (noisy) hard labels while providing more accurate confidence estimates.