# High Risk of Political Bias in Black Box Emotion Inference Models

arXiv: https://arxiv.org/abs/2407.13891 · HTML: https://mlsanity.com/paper/2407.13891/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 58 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 4 of 4)

- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [ChatGPT: Jack of all trades, master of none](https://mlsanity.com/paper/2302.10724/) (2023)
- [Public Sentiment Toward Solar Energy: Opinion Mining of Twitter Using a Transformer-Based Language Model](https://mlsanity.com/paper/2007.13306/) (2020)
- [Predicting Emotion Intensity in Polish Political Texts: Comparing Supervised Models and Large Language Models in a Resource-Poor Language](https://mlsanity.com/paper/2407.12141/) (2024)

## Built on by (most cited first, 2 of 2)

- [Political Alignment in Large Language Models: A Multidimensional Audit of Psychometric Identity and Behavioral Bias](https://mlsanity.com/paper/2601.06194/) (2026)
- [Retrieval Sensitivity to Identity Signals in Queries](https://mlsanity.com/paper/2609.36534/) (2026)

## Abstract (arXiv)

This paper investigates the presence of political bias in emotion inference models used for sentiment analysis (SA) in social science research. Machine learning models often reflect biases in their training data, impacting the validity of their outcomes. While previous research has highlighted gender and race biases, our study focuses on political bias - an underexplored yet pervasive issue that can skew the interpretation of text data across a wide array of studies. We conducted a bias audit on a Polish sentiment analysis model developed in our lab. By analyzing valence predictions for names and sentences involving Polish politicians, we uncovered systematic differences influenced by political affiliations. Our findings indicate that annotations by human raters propagate political biases into the model's predictions. To mitigate this, we pruned the training dataset of texts mentioning these politicians and observed a reduction in bias, though not its complete elimination. Given the significant implications of political bias in SA, our study emphasizes caution in employing these models for social science research. We recommend a critical examination of SA results and propose using lexicon-based systems as a more ideologically neutral alternative. This paper underscores the necessity for ongoing scrutiny and methodological adjustments to ensure the reliability and impartiality of the use of machine learning in academic and applied contexts.