# Vox Populi, Vox AI? Using Language Models to Estimate German Public Opinion

arXiv: https://arxiv.org/abs/2407.08563 · HTML: https://mlsanity.com/paper/2407.08563/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 93 references that are in the map.
- First posted on arXiv 11 Jul 2024.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale](https://mlsanity.com/paper/2211.03759/) (2022)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation](https://mlsanity.com/paper/2301.01768/) (2023)
- [Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve](https://mlsanity.com/paper/2309.13638/) (2023)
- [Questioning the Survey Responses of Large Language Models](https://mlsanity.com/paper/2306.07951/) (2023)
- ["My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models](https://mlsanity.com/paper/2402.14499/) (2024)
- [Multilingual Language Models are not Multicultural: A Case Study in Emotion](https://mlsanity.com/paper/2307.01370/) (2023)
- [Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models](https://mlsanity.com/paper/2310.12481/) (2023)
- [Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models](https://mlsanity.com/paper/2406.09289/) (2024)
- [This Land is {Your, My} Land: Evaluating Geopolitical Biases in Language Models](https://mlsanity.com/paper/2305.14610/) (2023)
- [Language Models Trained on Media Diets Can Predict Public Opinion](https://mlsanity.com/paper/2303.16779/) (2023)
- [DataTales: Investigating the use of Large Language Models for Authoring Data-Driven Articles](https://mlsanity.com/paper/2308.04076/) (2023)
- [Data Representativity for Machine Learning and AI Systems](https://mlsanity.com/paper/2203.04706/) (2022)
- [BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning](https://mlsanity.com/paper/2406.17764/) (2024)
- [Towards Coding Social Science Datasets with Language Models](https://mlsanity.com/paper/2306.02177/) (2023)

## Built on by (most cited first, 13 of 13)

- [Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models](https://mlsanity.com/paper/2510.11586/) (2025)
- [United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections](https://mlsanity.com/paper/2409.09045/) (2024)
- [German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies](https://mlsanity.com/paper/2511.21722/) (2025)
- [Extracting Affect Aggregates from Longitudinal Social Media Data with Temporal Adapters for Large Language Models](https://mlsanity.com/paper/2409.17990/) (2024)
- [Evaluating Cooperation in LLM Social Groups through Elected Leadership](https://mlsanity.com/paper/2604.11721/) (2026)
- [Learning from Convenience Samples: A Case Study on Fine-Tuning LLMs for Survey Non-response in the German Longitudinal Election Study](https://mlsanity.com/paper/2509.25063/) (2025)
- [Stochastic Parrots or Singing in Harmony? Testing Five Leading LLMs for their Ability to Replicate a Human Survey with Synthetic Data](https://mlsanity.com/paper/2603.00059/) (2026)
- [Graph-Based Alternatives to LLMs for Human Simulation](https://mlsanity.com/paper/2511.02135/) (2025)
- [Large language models accurately predict public perceptions of support for climate action worldwide](https://mlsanity.com/paper/2601.20141/) (2026)
- [People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe](https://mlsanity.com/paper/2608.07367/) (2026)
- [AI, Humans, and Data Science: Optimizing Roles Across Workflows and the Workforce](https://mlsanity.com/paper/2507.11597/) (2025)
- [Detecting undisclosed LLM-generated content in parliamentary texts](https://mlsanity.com/paper/2606.14209/) (2026)
- [Marginal Fidelity Does Not Establish User Simulation in Demographic Synthetic Survey Panels: Response Contracts, Support Collapse and Conditioning Failure](https://mlsanity.com/paper/2609.07305/) (2026)

## Abstract (arXiv)

The recent development of large language models (LLMs) has spurred discussions about whether LLM-generated "synthetic samples" could complement or replace traditional surveys, considering their training data potentially reflects attitudes and behaviors prevalent in the population. A number of mostly US-based studies have prompted LLMs to mimic survey respondents, with some of them finding that the responses closely match the survey data. However, several contextual factors related to the relationship between the respective target population and LLM training data might affect the generalizability of such findings. In this study, we investigate the extent to which LLMs can estimate public opinion in Germany, using the example of vote choice. We generate a synthetic sample of personas matching the individual characteristics of the 2017 German Longitudinal Election Study respondents. We ask the LLM GPT-3.5 to predict each respondent's vote choice and compare these predictions to the survey-based estimates on the aggregate and subgroup levels. We find that GPT-3.5 does not predict citizens' vote choice accurately, exhibiting a bias towards the Green and Left parties. While the LLM captures the tendencies of "typical" voter subgroups, such as partisans, it misses the multifaceted factors swaying individual voter choices. By examining the LLM-based prediction of voting behavior in a new context, our study contributes to the growing body of research about the conditions under which LLMs can be leveraged for studying public opinion. The findings point to disparities in opinion representation in LLMs and underscore the limitations in applying them for public opinion estimation.