# Distributions In, Distributions Out: The Case for Soft-Label Training

arXiv: https://arxiv.org/abs/2511.14117 · HTML: https://mlsanity.com/paper/2511.14117/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 18 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [What Can We Learn from Collective Human Opinions on Natural Language Inference Data?](https://mlsanity.com/paper/2010.03532/) (2020)
- [Label Distribution Learning](https://mlsanity.com/paper/1408.6027/) (2014)
- [Deep learning from crowds](https://mlsanity.com/paper/1709.01779/) (2017)
- [When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset](https://mlsanity.com/paper/2306.06826/) (2023)
- [Don't Waste a Single Annotation: Improving Single-Label Classifiers Through Soft Labels](https://mlsanity.com/paper/2311.05265/) (2023)
- [Learning From Biased Soft Labels](https://mlsanity.com/paper/2302.08155/) (2023)
- [Learning with Confidence: Training Better Classifiers from Soft Labels](https://mlsanity.com/paper/2409.16071/) (2024)
- [A Theoretical Analysis of Soft-Label vs Hard-Label Training in Neural Networks](https://mlsanity.com/paper/2412.09579/) (2024)

## Built on by (most cited first, 2 of 2)

- [SNCE: Geometry-Aware Supervision for Scalable Discrete Image Generation](https://mlsanity.com/paper/2603.15150/) (2026)
- [Metric-Dependent Annotation Saturation for Learning from Label Distributions](https://mlsanity.com/paper/2605.29797/) (2026)

## Abstract (arXiv)

Supervised classifiers output a distribution over classes but are typically trained against a single label obtained by collapsing multiple annotators into a majority vote. On tasks where annotator disagreement reflects genuine ambiguity -- natural language inference, politeness, visually ambiguous categorization -- this collapse discards information and forces models to express uniform confidence on inputs where humans systematically disagree. We compare soft-label training, which uses the full annotation distribution as the target, against hard-label training across three datasets spanning vision and NLP (ChaosNLI, POPQUORN, CIFAR-10H). Soft-label training matches or exceeds hard-label accuracy on every dataset, reduces KL divergence to the annotator distribution by 32% on average (p < 10^-4), and produces predictions whose per-sample entropy correlates 61% more strongly with annotator entropy -- models trained on distributions are uncertain precisely where humans are. We argue these benefits follow from a basic observation: when annotators legitimately disagree, the annotation distribution is the correct learning target, not a noisy estimate of it.