# Whose Ground Truth? Embracing Ambiguity in Human-Centered AI

arXiv: https://arxiv.org/abs/2610.10805 · HTML: https://mlsanity.com/paper/2610.10805/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Builds on (most cited first, 22 of 22)

- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [What Can We Learn from Collective Human Opinions on Natural Language Inference Data?](https://mlsanity.com/paper/2010.03532/) (2020)
- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [Stop Measuring Calibration When Humans Disagree](https://mlsanity.com/paper/2210.16133/) (2022)
- [When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset](https://mlsanity.com/paper/2306.06826/) (2023)
- [Analyzing Dataset Annotation Quality Management in the Wild](https://mlsanity.com/paper/2307.08153/) (2023)
- [Embracing Ambiguity: Shifting the Training Target of NLI Models](https://mlsanity.com/paper/2106.03020/) (2021)
- ["Seeing the Big through the Small": Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations?](https://mlsanity.com/paper/2406.17600/) (2024)
- [The Ambiguous World of Emotion Representation](https://mlsanity.com/paper/1909.00360/) (2019)
- [D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation](https://mlsanity.com/paper/2404.10857/) (2024)
- [AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels](https://mlsanity.com/paper/2410.22506/) (2024)
- [A Rose by Any Other Name: LLM-Generated Explanations Are Good Proxies for Human Explanations to Collect Label Distributions on NLI](https://mlsanity.com/paper/2412.13942/) (2024)
- [Rethinking Emotion Annotations in the Era of Large Language Models](https://mlsanity.com/paper/2412.07906/) (2024)
- [NUTMEG: Separating Signal From Noise in Annotator Disagreement](https://mlsanity.com/paper/2507.18890/) (2025)
- [Dual-Constrained Dynamical Neural ODEs for Ambiguity-aware Continuous Emotion Prediction](https://mlsanity.com/paper/2407.21344/) (2024)
- [From Disagreement to Understanding: The Case for Ambiguity Detection in NLI](https://mlsanity.com/paper/2507.15114/) (2025)
- [AmbER2: Dual Ambiguity-Aware Emotion Recognition Applied to Speech and Text](https://mlsanity.com/paper/2601.18010/) (2026)
- [Attention-Aware Answers of the Crowd](https://mlsanity.com/paper/1912.11238/) (2019)
- [Decoding Ambiguous Emotions with Test-Time Scaling in Audio-Language Models](https://mlsanity.com/paper/2602.03873/) (2026)
- [SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs](https://mlsanity.com/paper/2606.05376/) (2026)

## Abstract (arXiv)

As AI systems increasingly interact with people and make decisions about them, understanding human interpretations becomes an important part of developing human-centered AI. Conventional machine learning and AI systems are largely developed under the assumption that a single definitive ground truth exists, with variability in human annotations often resolved through aggregation or treated as noise. However, for many human-centered tasks, human interpretation is inherently ambiguous, and multiple interpretations of the same input may be simultaneously reasonable and valid. Reducing such ambiguity to a single target risks overlooking meaningful information about the diversity of human perception, judgment, and experience. In this position paper, we call for a shift towards modeling the interpretation space of plausible human judgments, while distinguishing meaningful ambiguity from annotation noise. We argue that this perspective should guide how AI systems are represented, learned, evaluated, deployed, and governed, supporting more human-centered AI that better reflects the diversity of human interpretation.