# Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification

arXiv: https://arxiv.org/abs/2407.17688 · HTML: https://mlsanity.com/paper/2407.17688/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 61 references that are in the map.
- First posted on arXiv 25 Jul 2024.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Benchmarking Cognitive Biases in Large Language Models as Evaluators](https://mlsanity.com/paper/2309.17012/) (2023)
- [Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration](https://mlsanity.com/paper/2305.13626/) (2023)
- [Debiasing Pre-trained Contextualised Embeddings](https://mlsanity.com/paper/2101.09523/) (2021)
- [How would Stance Detection Techniques Evolve after the Launch of ChatGPT?](https://mlsanity.com/paper/2212.14548/) (2022)
- [In Plain Sight: Media Bias Through the Lens of Factual Reporting](https://mlsanity.com/paper/1909.02670/) (2019)
- [The Self-Perception and Political Biases of ChatGPT](https://mlsanity.com/paper/2304.07333/) (2023)
- [Automated stance detection in complex topics and small languages: the challenging case of immigration in polarizing news media](https://mlsanity.com/paper/2305.13047/) (2023)
- [Pro or Anti? A Social Influence Model of Online Stance Flipping](https://mlsanity.com/paper/2106.11076/) (2021)

## Built on by (most cited first, 9 of 9)

- [Persuasion Dynamics in LLMs: Investigating Robustness and Adaptability in Knowledge and Safety with DuET-PD](https://mlsanity.com/paper/2508.17450/) (2025)
- [Large Language Models for Subjective Language Understanding: A Survey](https://mlsanity.com/paper/2508.07959/) (2025)
- [Read Between the Lines: A Benchmark for Uncovering Political Bias in Bangla News Articles](https://mlsanity.com/paper/2510.03898/) (2025)
- [Ideology-Based LLMs for Content Moderation](https://mlsanity.com/paper/2510.25805/) (2025)
- [Stylistic Evolution and LLM Neutrality in Singlish Language](https://mlsanity.com/paper/2601.06580/) (2026)
- [Adversarial Alignment: Ensuring Value Consistency in Large Language Models for Sensitive Domains](https://mlsanity.com/paper/2601.13137/) (2026)
- [TruthStance: An Annotated Dataset of Conversations on Truth Social](https://mlsanity.com/paper/2602.14406/) (2026)
- [Invisible Influences: Investigating Implicit Intersectional Biases through Persona Engineering in Large Language Models](https://mlsanity.com/paper/2604.06213/) (2026)
- [Retrieval Sensitivity to Identity Signals in Queries](https://mlsanity.com/paper/2609.36534/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable capabilities in executing tasks based on natural language queries. However, these models, trained on curated datasets, inherently embody biases ranging from racial to national and gender biases. It remains uncertain whether these biases impact the performance of LLMs for certain tasks. In this study, we investigate the political biases of LLMs within the stance classification task, specifically examining whether these models exhibit a tendency to more accurately classify politically-charged stances. Utilizing three datasets, seven LLMs, and four distinct prompting schemes, we analyze the performance of LLMs on politically oriented statements and targets. Our findings reveal a statistically significant difference in the performance of LLMs across various politically oriented stance classification tasks. Furthermore, we observe that this difference primarily manifests at the dataset level, with models and prompting schemes showing statistically similar performances across different stance classification datasets. Lastly, we observe that when there is greater ambiguity in the target the statement is directed towards, LLMs have poorer stance classification accuracy. Code & Dataset: http://doi.org/10.5281/zenodo.12938478