# Whose Side Are You On? Investigating the Political Stance of Large Language Models

arXiv: https://arxiv.org/abs/2403.13840 · HTML: https://mlsanity.com/paper/2403.13840/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 4 of 4)

- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootings](https://mlsanity.com/paper/1904.01596/) (2019)
- [Evaluation Metrics for Measuring Bias in Search Engine Results](https://mlsanity.com/paper/2210.10517/) (2022)

## Built on by (most cited first, 10 of 10)

- [Unmasking Conversational Bias in AI Multiagent Systems](https://mlsanity.com/paper/2501.14844/) (2025)
- [A Simulation System Towards Solving Societal-Scale Manipulation](https://mlsanity.com/paper/2410.13915/) (2024)
- [Large Language Models for Subjective Language Understanding: A Survey](https://mlsanity.com/paper/2508.07959/) (2025)
- [Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models](https://mlsanity.com/paper/2508.02360/) (2025)
- [Think Again! The Effect of Test-Time Compute on Preferences, Opinions, and Beliefs of Large Language Models](https://mlsanity.com/paper/2505.19621/) (2025)
- ["Amazing, They All Lean Left" -- Analyzing the Political Temperaments of Current LLMs](https://mlsanity.com/paper/2507.08027/) (2025)
- [Identifying the sources of ideological bias in GPT models through linguistic variation in output](https://mlsanity.com/paper/2409.06043/) (2024)
- [Agent-Enhanced Large Language Models for Researching Political Institutions](https://mlsanity.com/paper/2503.13524/) (2025)
- [Social media polarization during conflict: Insights from an ideological stance dataset on Israel-Palestine Reddit comments](https://mlsanity.com/paper/2502.00414/) (2025)
- [Measuring Political Stance and Consistency in Large Language Models](https://mlsanity.com/paper/2601.17016/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have gained significant popularity for their application in various everyday tasks such as text generation, summarization, and information retrieval. As the widespread adoption of LLMs continues to surge, it becomes increasingly crucial to ensure that these models yield responses that are politically impartial, with the aim of preventing information bubbles, upholding fairness in representation, and mitigating confirmation bias. In this paper, we propose a quantitative framework and pipeline designed to systematically investigate the political orientation of LLMs. Our investigation delves into the political alignment of LLMs across a spectrum of eight polarizing topics, spanning from abortion to LGBTQ issues. Across topics, the results indicate that LLMs exhibit a tendency to provide responses that closely align with liberal or left-leaning perspectives rather than conservative or right-leaning ones when user queries include details pertaining to occupation, race, or political affiliation. The findings presented in this study not only reaffirm earlier observations regarding the left-leaning characteristics of LLMs but also surface particular attributes, such as occupation, that are particularly susceptible to such inclinations even when directly steered towards conservatism. As a recommendation to avoid these models providing politicised responses, users should be mindful when crafting queries, and exercise caution in selecting neutral prompt language.