# SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs

arXiv: https://arxiv.org/abs/2606.05376 · HTML: https://mlsanity.com/paper/2606.05376/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Builds on (most cited first, 22 of 22)

- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model](https://mlsanity.com/paper/2503.24290/) (2025)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [What Can We Learn from Collective Human Opinions on Natural Language Inference Data?](https://mlsanity.com/paper/2010.03532/) (2020)
- [Stop Measuring Calibration When Humans Disagree](https://mlsanity.com/paper/2210.16133/) (2022)
- [Benchmarking Distributional Alignment of Large Language Models](https://mlsanity.com/paper/2411.05403/) (2024)
- [Can Large Language Models Capture Dissenting Human Voices?](https://mlsanity.com/paper/2305.13788/) (2023)
- [Distributed NLI: Learning to Predict Human Opinion Distributions for Language Reasoning](https://mlsanity.com/paper/2104.08676/) (2021)
- [Embracing Ambiguity: Shifting the Training Target of NLI Models](https://mlsanity.com/paper/2106.03020/) (2021)
- ["Seeing the Big through the Small": Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations?](https://mlsanity.com/paper/2406.17600/) (2024)
- [The Ambiguous World of Emotion Representation](https://mlsanity.com/paper/1909.00360/) (2019)
- [Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement](https://mlsanity.com/paper/2502.06207/) (2025)
- [AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels](https://mlsanity.com/paper/2410.22506/) (2024)
- [A Rose by Any Other Name: LLM-Generated Explanations Are Good Proxies for Human Explanations to Collect Label Distributions on NLI](https://mlsanity.com/paper/2412.13942/) (2024)
- [Arbiters of Ambivalence: Challenges of Using LLMs in No-Consensus Tasks](https://mlsanity.com/paper/2505.23820/) (2025)
- [Dual-Constrained Dynamical Neural ODEs for Ambiguity-aware Continuous Emotion Prediction](https://mlsanity.com/paper/2407.21344/) (2024)
- [From Disagreement to Understanding: The Case for Ambiguity Detection in NLI](https://mlsanity.com/paper/2507.15114/) (2025)
- [AmbER2: Dual Ambiguity-Aware Emotion Recognition Applied to Speech and Text](https://mlsanity.com/paper/2601.18010/) (2026)
- [Decoding Ambiguous Emotions with Test-Time Scaling in Audio-Language Models](https://mlsanity.com/paper/2602.03873/) (2026)

## Built on by (most cited first, 2 of 2)

- [Whose Ground Truth? Embracing Ambiguity in Human-Centered AI](https://mlsanity.com/paper/2610.10805/) (2026)
- [Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders](https://mlsanity.com/paper/2610.17771/) (2026)

## Abstract (arXiv)

Many human-centered tasks, including natural language inference (NLI) and emotion recognition (ER), have multiple plausible interpretations, leading to label ambiguity and challenging disagreements across human annotators. As LLMs are increasingly deployed in real-world settings, faithfully modeling such ambiguity is essential to identify contested inputs, preserve variability in ambiguous cases, and capture the full distribution of human judgments. Yet, existing LLM alignment approaches have predominantly assumed a single correct label, excluding annotator disagreement during optimization. Instead of treating this ambiguity as noise, we show how to treat it as information that improves model behavior through a new algorithm called SMARTLY HANDLING AMBIGUOUS LABELS IN ALIGNING LLMS (SHALA-LLM). This reinforcement learning framework provides a new way for LLMs to learn directly from annotator distributions while dynamically prioritizing highly ambiguous samples during optimization. Experiments on ambiguity-sensitive NLI and ER benchmarks, including ChaosNLI, GoEmotions, and MSP-Podcast, demonstrate that SHALA-LLM improves agreement with annotator label distributions, e.g. on ChaosNLI, it reduces Jensen-Shannon Distance by up to 62.1%. At the same time, SHALA-LLM improves F1 by up to 16.7%, showing that modeling annotator disagreement can also strengthen classification performance.