# Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study arXiv: https://arxiv.org/abs/2412.13169 ยท HTML: https://mlsanity.com/paper/2412.13169/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Builds on 18 of its 46 references that are in the map. - First posted on arXiv 17 Dec 2024. ## Builds on (most cited first, 18 of 18) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023) - [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023) - [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022) - [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023) - [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022) - [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024) - [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024) - [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023) - [Do LLMs exhibit human-like response biases? A case study in survey design](https://mlsanity.com/paper/2311.04076/) (2023) - [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024) - [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022) - [Aligning Language Models to User Opinions](https://mlsanity.com/paper/2305.14929/) (2023) - [Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Information](https://mlsanity.com/paper/2402.18144/) (2024) - [You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments](https://mlsanity.com/paper/2311.09718/) (2023) - [Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in](https://mlsanity.com/paper/2404.18460/) (2024) - [The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models](https://mlsanity.com/paper/2406.11096/) (2024) ## Built on by (most cited first, 14 of 14) - [Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges](https://mlsanity.com/paper/2502.12378/) (2025) - [Many AI Analysts, One Dataset: Navigating the Agentic Data Science Multiverse](https://mlsanity.com/paper/2602.18710/) (2026) - [Don't Trust Generative Agents to Mimic Communication on Social Networks Unless You Benchmarked their Empirical Realism](https://mlsanity.com/paper/2506.21974/) (2025) - [German General Social Survey Personas: A Survey-Derived Persona Prompt Collection for Population-Aligned LLM Studies](https://mlsanity.com/paper/2511.21722/) (2025) - [Guided Persona-based AI Surveys: Can we replicate personal mobility preferences at scale using LLMs?](https://mlsanity.com/paper/2501.13955/) (2025) - [Tastes without distinction: silicon samples and the synthetic construction of tastes](https://mlsanity.com/paper/2606.30085/) (2026) - [Emulating Public Opinion: A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case](https://mlsanity.com/paper/2509.09871/) (2025) - [Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation](https://mlsanity.com/paper/2510.13884/) (2025) - [Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models](https://mlsanity.com/paper/2506.11798/) (2025) - [Measure what Matters: Psychometric Evaluation of AI with Situational Judgment Tests](https://mlsanity.com/paper/2510.22170/) (2025) - [Moral Lenses, Political Coordinates: Towards Ideological Positioning of Morally Conditioned LLMs](https://mlsanity.com/paper/2601.08634/) (2026) - [Does Topic Sentiment Cause Perceived Ideology? Comparing Human and LLM Annotations in Political News Articles](https://mlsanity.com/paper/2606.06715/) (2026) - [Multimodal Large Language Models as Synthetic Participants in Video-Based Studies: An Evaluation](https://mlsanity.com/paper/2606.07541/) (2026) - [Cultural Divergence Preservation: Diagnosing Flattening and Caricature in LLM-Simulated Survey Populations](https://mlsanity.com/paper/2609.29928/) (2026) ## Abstract (arXiv) In recent research, large language models (LLMs) have been increasingly used to investigate public opinions. This study investigates the algorithmic fidelity of LLMs, i.e., the ability to replicate the socio-cultural context and nuanced opinions of human participants. Using open-ended survey data from the German Longitudinal Election Studies (GLES), we prompt different LLMs to generate synthetic public opinions reflective of German subpopulations by incorporating demographic features into the persona prompts. Our results show that Llama performs better than other LLMs at representing subpopulations, particularly when there is lower opinion diversity within those groups. Our findings further reveal that the LLM performs better for supporters of left-leaning parties like The Greens and The Left compared to other parties, and matches the least with the right-party AfD. Additionally, the inclusion or exclusion of specific variables in the prompts can significantly impact the models' predictions. These findings underscore the importance of aligning LLMs to more effectively model diverse public opinions while minimizing political biases and enhancing robustness in representativeness.