mlsanityData to 14 Oct 2026

Synthesizing Public Opinions with LLMs: Role Creation, Impacts, and the Future to eDemorcacy

arXiv 2504.00241 ↗First posted 31 Mar 2025cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

This paper investigates the use of Large Language Models (LLMs) to synthesize public opinion data, addressing challenges in traditional survey methods like declining response rates and non-response bias. We introduce a novel technique: role creation based on knowledge injection, a form of in-context learning that leverages RAG and specified personality profiles from the HEXACO model and demographic information, and uses that for dynamically generated prompts. This method allows LLMs to simulate diverse opinions more accurately than existing prompt engineering approaches. We compare our results with pre-trained models with standard few-shot prompts. Experiments using questions from the Cooperative Election Study (CES) demonstrate that our role-creation approach significantly improves the alignment of LLM-generated opinions with real-world human survey responses, increasing answer adherence. In addition, we discuss challenges, limitations and future research directions.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 27 of its 58 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

27 of 58 references.

References · 27

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  3. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  4. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  5. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  6. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 260
  7. 2023Can Large Language Models Transform Computational Social Science?cited by 250
  8. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 249
  9. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 220
  10. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 215
  11. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 199
  12. 2021KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extractioncited by 180
15 more
  1. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 178
  2. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  3. 2023CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agentscited by 65
  4. 2024Can Large Language Model Agents Simulate Human Trust Behavior?cited by 64
  5. 2020Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformerscited by 57
  6. 2023Do LLM Agents Exhibit Social Behavior?cited by 53
  7. 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
  8. 2023Large Language Models Can Be Used to Estimate the Latent Positions of Politicianscited by 33
  9. 2024LLMs generate structurally realistic social networks but overestimate political homophilycited by 33
  10. 2023Accuracy and Political Bias of News Source Credibility Ratings by Large Language Modelscited by 25
  11. 2024ChatGPT as Research Scientist: Probing GPT's Capabilities as a Research Librarian, Research Ethicist, Data Generator and Data Predictorcited by 21
  12. 2023Large Language Models as Subpopulation Representative Models: A Reviewcited by 16
  13. 2024Representation Bias in Political Sample Simulations with Large Language Modelscited by 10
  14. 2023Can Large Language Models Capture Public Opinion about Global Warming? An Empirical Assessment of Algorithmic Fidelity and Biascited by 9
  15. 2024Diverse, but Divisive: LLMs Can Exaggerate Gender Differences in Opinion Related to Harms of Misinformationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Can Reasoning Help Large Language Models Capture Human Annotator Disagreement?cited by 10
  2. 2026Parametric Social Identity Injection and Diversification in Public Opinion Simulationcited by 2
  3. 2025Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulationcited by 1
  4. 2026Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogatescited by 1
  5. 2025Recalibrating the Compass: Integrating Large Language Models into Classical Research Methodscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction
  2. 2025An Analysis of Large Language Models for Simulating User Responses in Surveys
  3. 2023Aligning Language Models to User Opinions
  4. 2025Emulating Public Opinion: A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case
  5. 2025From Delegates to Trustees: How Optimizing for Long-Term Interests Shapes Bias and Alignment in LLM
  6. 2024GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy
  7. 2024LLM Voting: Human Choices and AI Collective Decision Making
  8. 2024Using LLMs to Model the Beliefs and Preferences of Targeted Populations