# United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections

arXiv: https://arxiv.org/abs/2409.09045 · HTML: https://mlsanity.com/paper/2409.09045/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 66 references that are in the map.
- First posted on arXiv 29 Aug 2024.

## Builds on (most cited first, 24 of 24)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation](https://mlsanity.com/paper/2301.01768/) (2023)
- [Do LLMs exhibit human-like response biases? A case study in survey design](https://mlsanity.com/paper/2311.04076/) (2023)
- [Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve](https://mlsanity.com/paper/2309.13638/) (2023)
- [Large language models that replace human participants can harmfully misportray and flatten identity groups](https://mlsanity.com/paper/2402.01908/) (2024)
- [Aligning Language Models to User Opinions](https://mlsanity.com/paper/2305.14929/) (2023)
- [The Political Preferences of LLMs](https://mlsanity.com/paper/2402.01789/) (2024)
- [Questioning the Survey Responses of Large Language Models](https://mlsanity.com/paper/2306.07951/) (2023)
- [AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction](https://mlsanity.com/paper/2305.09620/) (2023)
- [Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models](https://mlsanity.com/paper/2310.12481/) (2023)
- [Can Large Language Models Capture Dissenting Human Voices?](https://mlsanity.com/paper/2305.13788/) (2023)
- [This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Models](https://mlsanity.com/paper/2305.14610/) (2023)
- [Language Models Trained on Media Diets Can Predict Public Opinion](https://mlsanity.com/paper/2303.16779/) (2023)
- [Easy Problems That LLMs Get Wrong](https://mlsanity.com/paper/2405.19616/) (2024)
- [Data Representativity for Machine Learning and AI Systems](https://mlsanity.com/paper/2203.04706/) (2022)
- [Demonstrations of the Potential of AI-based Political Issue Polling](https://mlsanity.com/paper/2307.04781/) (2023)
- [Vox Populi, Vox AI? Using Language Models to Estimate German Public Opinion](https://mlsanity.com/paper/2407.08563/) (2024)
- [Representation Bias in Political Sample Simulations with Large Language Models](https://mlsanity.com/paper/2407.11409/) (2024)
- [How Different Is Stereotypical Bias Across Languages?](https://mlsanity.com/paper/2307.07331/) (2023)

## Built on by (most cited first, 5 of 5)

- [LLM-Based Social Simulations Require a Boundary](https://mlsanity.com/paper/2506.19806/) (2025)
- [Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction](https://mlsanity.com/paper/2502.16280/) (2025)
- [CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Models](https://mlsanity.com/paper/2512.07890/) (2025)
- [Characterizing the ability of LLMs to recapitulate Americans' distributional responses to public opinion polling questions across political issues](https://mlsanity.com/paper/2603.20229/) (2026)
- [Large Language Models as Implicit Sociological Models: Reconstructing Voting Behaviour from Sociodemographic Profiles](https://mlsanity.com/paper/2608.15871/) (2026)

## Abstract (arXiv)

"Synthetic samples" based on large language models (LLMs) have been argued to serve as efficient alternatives to surveys of humans, assuming that their training data includes information on human attitudes and behavior. However, LLM-synthetic samples might exhibit bias, for example due to training data and fine-tuning processes being unrepresentative of diverse contexts. Such biases risk reinforcing existing biases in research, policymaking, and society. Therefore, researchers need to investigate if and under which conditions LLM-generated synthetic samples can be used for public opinion prediction. In this study, we examine to what extent LLM-based predictions of individual public opinion exhibit context-dependent biases by predicting the results of the 2024 European Parliament elections. Prompting three LLMs with individual-level background information of 26,000 eligible European voters, we ask the LLMs to predict each person's voting behavior. By comparing them to the actual results, we show that LLM-based predictions of future voting behavior largely fail, their accuracy is unequally distributed across national and linguistic contexts, and they require detailed attitudinal information in the prompt. The findings emphasize the limited applicability of LLM-synthetic samples to public opinion prediction. In investigating their contextual biases, this study contributes to the understanding and mitigation of inequalities in the development of LLMs and their applications in computational social science.