# Synthesizing Public Opinions with LLMs: Role Creation, Impacts, and the Future to eDemorcacy

arXiv: https://arxiv.org/abs/2504.00241 · HTML: https://mlsanity.com/paper/2504.00241/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 58 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Builds on (most cited first, 27 of 27)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models](https://mlsanity.com/paper/2205.10770/) (2022)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs](https://mlsanity.com/paper/2311.04892/) (2023)
- [KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction](https://mlsanity.com/paper/2104.07650/) (2021)
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](https://mlsanity.com/paper/2312.05934/) (2023)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agents](https://mlsanity.com/paper/2310.17512/) (2023)
- [Can Large Language Model Agents Simulate Human Trust Behavior?](https://mlsanity.com/paper/2402.04559/) (2024)
- [Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers](https://mlsanity.com/paper/2005.11787/) (2020)
- [Do LLM Agents Exhibit Social Behavior?](https://mlsanity.com/paper/2312.15198/) (2023)
- [When Large Language Models Meet Vector Databases: A Survey](https://mlsanity.com/paper/2402.01763/) (2024)
- [Large Language Models Can Be Used to Estimate the Latent Positions of Politicians](https://mlsanity.com/paper/2303.12057/) (2023)
- [LLMs generate structurally realistic social networks but overestimate political homophily](https://mlsanity.com/paper/2408.16629/) (2024)
- [Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models](https://mlsanity.com/paper/2304.00228/) (2023)
- [ChatGPT as Research Scientist: Probing GPT's Capabilities as a Research Librarian, Research Ethicist, Data Generator and Data Predictor](https://mlsanity.com/paper/2406.14765/) (2024)
- [Large Language Models as Subpopulation Representative Models: A Review](https://mlsanity.com/paper/2310.17888/) (2023)
- [Representation Bias in Political Sample Simulations with Large Language Models](https://mlsanity.com/paper/2407.11409/) (2024)
- [Can Large Language Models Capture Public Opinion about Global Warming? An Empirical Assessment of Algorithmic Fidelity and Bias](https://mlsanity.com/paper/2311.00217/) (2023)
- [Diverse, but Divisive: LLMs Can Exaggerate Gender Differences in Opinion Related to Harms of Misinformation](https://mlsanity.com/paper/2401.16558/) (2024)

## Built on by (most cited first, 5 of 5)

- [Can Reasoning Help Large Language Models Capture Human Annotator Disagreement?](https://mlsanity.com/paper/2506.19467/) (2025)
- [Parametric Social Identity Injection and Diversification in Public Opinion Simulation](https://mlsanity.com/paper/2603.16142/) (2026)
- [Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation](https://mlsanity.com/paper/2510.13884/) (2025)
- [Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates](https://mlsanity.com/paper/2608.29455/) (2026)
- [Recalibrating the Compass: Integrating Large Language Models into Classical Research Methods](https://mlsanity.com/paper/2505.19402/) (2025)

## Abstract (arXiv)

This paper investigates the use of Large Language Models (LLMs) to synthesize public opinion data, addressing challenges in traditional survey methods like declining response rates and non-response bias. We introduce a novel technique: role creation based on knowledge injection, a form of in-context learning that leverages RAG and specified personality profiles from the HEXACO model and demographic information, and uses that for dynamically generated prompts. This method allows LLMs to simulate diverse opinions more accurately than existing prompt engineering approaches. We compare our results with pre-trained models with standard few-shot prompts. Experiments using questions from the Cooperative Election Study (CES) demonstrate that our role-creation approach significantly improves the alignment of LLM-generated opinions with real-world human survey responses, increasing answer adherence. In addition, we discuss challenges, limitations and future research directions.