# Political Bias in LLMs: Unaligned Moral Values in Agent-centric Simulations

arXiv: https://arxiv.org/abs/2408.11415 · HTML: https://mlsanity.com/paper/2408.11415/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 21 Aug 2024.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 11 of 11)

- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [The Self-Perception and Political Biases of ChatGPT](https://mlsanity.com/paper/2304.07333/) (2023)
- [Simulacra as Conscious Exotica](https://mlsanity.com/paper/2402.12422/) (2024)
- [NIR-Prompt: A Multi-task Generalized Neural Information Retrieval Training Framework](https://mlsanity.com/paper/2212.00229/) (2022)

## Built on by (most cited first, 5 of 5)

- [Don't Trust Generative Agents to Mimic Communication on Social Networks Unless You Benchmarked their Empirical Realism](https://mlsanity.com/paper/2506.21974/) (2025)
- [Made-in China, Thinking in America:U.S. Values Persist in Chinese LLMs](https://mlsanity.com/paper/2512.13723/) (2025)
- [Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires](https://mlsanity.com/paper/2507.10073/) (2025)
- [Moral Lenses, Political Coordinates: Towards Ideological Positioning of Morally Conditioned LLMs](https://mlsanity.com/paper/2601.08634/) (2026)
- [Next Reply Prediction X Dataset: Linguistic Discrepancies in Naively Generated Content](https://mlsanity.com/paper/2602.19177/) (2026)

## Abstract (arXiv)

Contemporary research in social sciences increasingly utilizes state-of-the-art generative language models to annotate or generate content. While these models achieve benchmark-leading performance on common language tasks, their application to novel out-of-domain tasks remains insufficiently explored. To address this gap, we investigate how personalized language models align with human responses on the Moral Foundation Theory Questionnaire. We adapt open-source generative language models to different political personas and repeatedly survey these models to generate synthetic data sets where model-persona combinations define our sub-populations. Our analysis reveals that models produce inconsistent results across multiple repetitions, yielding high response variance. Furthermore, the alignment between synthetic data and corresponding human data from psychological studies shows a weak correlation, with conservative persona-prompted models particularly failing to align with actual conservative populations. These results suggest that language models struggle to coherently represent ideologies through in-context prompting due to their alignment process. Thus, using language models to simulate social interactions requires measurable improvements in in-context optimization or parameter manipulation to align with psychological and sociological stereotypes properly.