# Representation Bias in Political Sample Simulations with Large Language Models

arXiv: https://arxiv.org/abs/2407.11409 · HTML: https://mlsanity.com/paper/2407.11409/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Builds on (most cited first, 21 of 21)

- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [S3: Social-network Simulation System with Large Language Model-Empowered Agents](https://mlsanity.com/paper/2307.14984/) (2023)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation](https://mlsanity.com/paper/2301.01768/) (2023)
- [Gender bias and stereotypes in Large Language Models](https://mlsanity.com/paper/2308.14921/) (2023)
- [Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech](https://mlsanity.com/paper/2302.07736/) (2023)
- [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://mlsanity.com/paper/2305.14456/) (2023)
- [A Survey on Fairness in Large Language Models](https://mlsanity.com/paper/2308.10149/) (2023)
- [Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks](https://mlsanity.com/paper/2304.10145/) (2023)
- [Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategies](https://mlsanity.com/paper/2304.07683/) (2023)
- [LLM-Rec: Personalized Recommendation via Prompting Large Language Models](https://mlsanity.com/paper/2307.15780/) (2023)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [What's in a Name? Auditing Large Language Models for Race and Gender Bias](https://mlsanity.com/paper/2402.14875/) (2024)
- [The illusion of artificial inclusion](https://mlsanity.com/paper/2401.08572/) (2024)
- [Public Perception of Generative AI on Twitter: An Empirical Study Based on Occupation and Usage](https://mlsanity.com/paper/2305.09537/) (2023)
- [Human vs. LMMs: Exploring the Discrepancy in Emoji Interpretation and Usage in Digital Communication](https://mlsanity.com/paper/2401.08212/) (2024)
- [Excitements and Concerns in the Post-ChatGPT Era: Deciphering Public Perception of AI through Social Media Analysis](https://mlsanity.com/paper/2307.05809/) (2023)

## Built on by (most cited first, 10 of 10)

- [LLM Generated Persona is a Promise with a Catch](https://mlsanity.com/paper/2503.16527/) (2025)
- [ElectionSim: Massive Population Election Simulation Powered by Large Language Model Driven Agents](https://mlsanity.com/paper/2410.20746/) (2024)
- [United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections](https://mlsanity.com/paper/2409.09045/) (2024)
- [Large Language Models in Politics and Democracy: A Comprehensive Survey](https://mlsanity.com/paper/2412.04498/) (2024)
- [Synthesizing Public Opinions with LLMs: Role Creation, Impacts, and the Future to eDemorcacy](https://mlsanity.com/paper/2504.00241/) (2025)
- [A Cross-Cultural Comparison of LLM-based Public Opinion Simulation: Evaluating Chinese and U.S. Models on Diverse Societies](https://mlsanity.com/paper/2506.21587/) (2025)
- [Assessing Historical Structural Oppression Worldwide via Rule-Guided Prompting of Large Language Models](https://mlsanity.com/paper/2509.15216/) (2025)
- [Improving Cross-Cultural Survey Simulation with Calibrated Value Personas](https://mlsanity.com/paper/2605.16193/) (2026)
- [What Helps Language Models Predict Human Beliefs: Demographics or Prior Stances?](https://mlsanity.com/paper/2511.18616/) (2025)
- [Reasoning Boosts Opinion Alignment in LLMs](https://mlsanity.com/paper/2603.01214/) (2026)

## Abstract (arXiv)

This study seeks to identify and quantify biases in simulating political samples with Large Language Models, specifically focusing on vote choice and public opinion. Using the GPT-3.5-Turbo model, we leverage data from the American National Election Studies, German Longitudinal Election Study, Zuobiao Dataset, and China Family Panel Studies to simulate voting behaviors and public opinions. This methodology enables us to examine three types of representation bias: disparities based on the the country's language, demographic groups, and political regime types. The findings reveal that simulation performance is generally better for vote choice than for public opinions, more accurate in English-speaking countries, more effective in bipartisan systems than in multi-partisan systems, and stronger in democratic settings than in authoritarian regimes. These results contribute to enhancing our understanding and developing strategies to mitigate biases in AI applications within the field of computational social science.