# End-to-End Annotator Bias Approximation on Crowdsourced Single-Label Sentiment Analysis

arXiv: https://arxiv.org/abs/2111.02326 · HTML: https://mlsanity.com/paper/2111.02326/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 3 Nov 2021.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Deep learning from crowds](https://mlsanity.com/paper/1709.01779/) (2017)
- [SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild](https://mlsanity.com/paper/1901.02839/) (2019)
- [Learning From Noisy Singly-labeled Data](https://mlsanity.com/paper/1712.04577/) (2017)
- [Learning from Crowds by Modeling Common Confusions](https://mlsanity.com/paper/2012.13052/) (2020)
- [Fast Dawid-Skene: A Fast Vote Aggregation Scheme for Sentiment Classification](https://mlsanity.com/paper/1803.02781/) (2018)
- [Learning with Noisy Labels for Sentence-level Sentiment Classification](https://mlsanity.com/paper/1909.00124/) (2019)
- [Learning from Imperfect Annotations](https://mlsanity.com/paper/2004.03473/) (2020)

## Built on by (most cited first, 4 of 4)

- [Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thought](https://mlsanity.com/paper/2401.06836/) (2024)
- [It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance](https://mlsanity.com/paper/2305.09022/) (2023)
- [Understanding Interpersonal Conflict Types and their Impact on Perception Classification](https://mlsanity.com/paper/2208.08758/) (2022)
- [EcoVerse: An Annotated Twitter Dataset for Eco-Relevance Classification, Environmental Impact Analysis, and Stance Detection](https://mlsanity.com/paper/2404.05133/) (2024)

## Abstract (arXiv)

Sentiment analysis is often a crowdsourcing task prone to subjective labels given by many annotators. It is not yet fully understood how the annotation bias of each annotator can be modeled correctly with state-of-the-art methods. However, resolving annotator bias precisely and reliably is the key to understand annotators' labeling behavior and to successfully resolve corresponding individual misconceptions and wrongdoings regarding the annotation task. Our contribution is an explanation and improvement for precise neural end-to-end bias modeling and ground truth estimation, which reduces an undesired mismatch in that regard of the existing state-of-the-art. Classification experiments show that it has potential to improve accuracy in cases where each sample is annotated only by one single annotator. We provide the whole source code publicly and release an own domain-specific sentiment dataset containing 10,000 sentences discussing organic food products. These are crawled from social media and are singly labeled by 10 non-expert annotators.