# Don't Blame the Data, Blame the Model: Understanding Noise and Bias When Learning from Subjective Annotations

arXiv: https://arxiv.org/abs/2403.04085 · HTML: https://mlsanity.com/paper/2403.04085/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 48 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Builds on (most cited first, 30 of 33)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [Identifying Mislabeled Data using the Area Under the Margin Ranking](https://mlsanity.com/paper/2001.10528/) (2020)
- [Data and its (dis)contents: A survey of dataset development and use in machine learning research](https://mlsanity.com/paper/2012.05345/) (2020)
- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels](https://mlsanity.com/paper/1905.05040/) (2019)
- [Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision](https://mlsanity.com/paper/1906.01620/) (2019)
- [Adversarial Filters of Dataset Biases](https://mlsanity.com/paper/2002.04108/) (2020)
- [Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples](https://mlsanity.com/paper/1704.07433/) (2017)
- [Continual Deep Learning by Functional Regularisation of Memorable Past](https://mlsanity.com/paper/2004.14070/) (2020)
- [A Walk with SGD](https://mlsanity.com/paper/1802.08770/) (2018)
- [Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreement](https://mlsanity.com/paper/2109.13563/) (2021)
- [Measuring a hate speech spectrum with faceted Rasch item response theory and perspective-aware, explainable-by-design deep learning](https://mlsanity.com/paper/2009.10277/) (2020)
- [Handling Bias in Toxic Speech Detection: A Survey](https://mlsanity.com/paper/2202.00126/) (2022)
- [Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information](https://mlsanity.com/paper/2301.05036/) (2023)
- [Capturing Perspectives of Crowdsourced Annotators in Subjective Learning Tasks](https://mlsanity.com/paper/2311.09743/) (2023)
- [Checkpoint Ensembles: Ensemble Methods from a Single Training Process](https://mlsanity.com/paper/1710.03282/) (2017)
- [DQI: Measuring Data Quality in NLP](https://mlsanity.com/paper/2005.00816/) (2020)
- [Active Learning for Interactive Neural Machine Translation of Data Streams](https://mlsanity.com/paper/1807.11243/) (2018)

## Built on by (most cited first, 10 of 10)

- [Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions](https://mlsanity.com/paper/2502.20897/) (2025)
- ["Seeing the Big through the Small": Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations?](https://mlsanity.com/paper/2406.17600/) (2024)
- [Label Convergence: Defining an Upper Performance Bound in Object Recognition through Contradictory Annotations](https://mlsanity.com/paper/2409.09412/) (2024)
- [A Collaborative Content Moderation Framework for Toxicity Detection based on Conformalized Estimates of Annotation Disagreement](https://mlsanity.com/paper/2411.04090/) (2024)
- [Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding](https://mlsanity.com/paper/2508.20765/) (2025)
- [Conflicts in Texts: Data, Implications and Challenges](https://mlsanity.com/paper/2504.19472/) (2025)
- [Are you sure? Measuring models bias in content moderation through uncertainty](https://mlsanity.com/paper/2509.22699/) (2025)
- [Behind Closed Words: Creating and Investigating the forePLay Annotated Dataset for Polish Erotic Discourse](https://mlsanity.com/paper/2412.17533/) (2024)
- [Are Humans as Brittle as Large Language Models?](https://mlsanity.com/paper/2509.07869/) (2025)
- [Learning Who Disagrees: Demographic Importance Weighting for Modeling Annotator Distributions with DiADEM](https://mlsanity.com/paper/2604.08425/) (2026)

## Abstract (arXiv)

Researchers have raised awareness about the harms of aggregating labels especially in subjective tasks that naturally contain disagreements among human annotators. In this work we show that models that are only provided aggregated labels show low confidence on high-disagreement data instances. While previous studies consider such instances as mislabeled, we argue that the reason the high-disagreement text instances have been hard-to-learn is that the conventional aggregated models underperform in extracting useful signals from subjective tasks. Inspired by recent studies demonstrating the effectiveness of learning from raw annotations, we investigate classifying using Multiple Ground Truth (Multi-GT) approaches. Our experiments show an improvement of confidence for the high-disagreement instances.