mlsanityData to 14 Oct 2026

Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction

arXiv 2502.16280 ↗First posted 22 Feb 2025cs.LG cs.AIcited by 5

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Generative AI (GenAI) is increasingly used in survey contexts to simulate human preferences. While many research endeavors evaluate the quality of synthetic GenAI data by comparing model-generated responses to gold-standard survey results, fundamental questions about the validity and reliability of using LLMs as substitutes for human respondents remain. Our study provides a technical analysis of how demographic attributes and prompt variations influence latent opinion mappings in large language models (LLMs) and evaluates their suitability for survey-based predictions. Using 14 different models, we find that LLM-generated data fails to replicate the variance observed in real-world human responses, particularly across demographic subgroups. In the political space, persona-to-party mappings exhibit limited differentiation, resulting in synthetic data that lacks the nuanced distribution of opinions found in survey data. Moreover, we show that prompt sensitivity can significantly alter outputs for some models, further undermining the stability and predictiveness of LLM-based simulations. As a key contribution, we adapt a probe-based methodology that reveals how LLMs encode political affiliations in their latent space, exposing the systematic distortions introduced by these models. Our findings highlight critical limitations in AI-generated survey data, urging caution in its use for public opinion research, social science experimentation, and computational behavioral modeling.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 22 of its 32 references that are in the map. First posted on arXiv 22 Feb 2025.

Builds on

22 of 32 references.

References · 22

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  3. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  4. 2023Whose Opinions Do Language Models Reflect?cited by 773
  5. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  6. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  7. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  8. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 310
  9. 2023Can Large Language Models Transform Computational Social Science?cited by 250
  10. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 249
  11. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  12. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 185
10 more
  1. 2024ProSA: Assessing and Understanding the Prompt Sensitivity of LLMscited by 149
  2. 2023ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learningcited by 100
  3. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 83
  4. 2023Questioning the Survey Responses of Large Language Modelscited by 74
  5. 2024Can Large Language Model Agents Simulate Human Trust Behavior?cited by 64
  6. 2024POSIX: A Prompt Sensitivity Index For Large Language Modelscited by 59
  7. 2023AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Predictioncited by 48
  8. 2023The language of prompting: What linguistic properties make a prompt successful?cited by 44
  9. 2024A Large-Scale Simulation on Large Language Models for Decision-Making in Political Sciencecited by 10
  10. 2024United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Electionscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
  2. 2025CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Modelscited by 5
  3. 2026Reasoning Boosts Opinion Alignment in LLMscited by 0
  4. 2026Where are the Humans? A Scoping Review of Fairness in Multi-agent AI Systemscited by 0
  5. 2026Appearing Legitimate is Not Enough: Interrogating Synthetic Agents in Representational Processes through a Participatory Design Lenscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reading Between the Tokens: Improving Preference Predictions through Mechanistic Forecasting ↗
  2. 2024LLM Voting: Human Choices and AI Collective Decision Making
  3. 2025Emulating Public Opinion: A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case
  4. 2026Unpacking Human Preference for LLMs: Demographically Aware Evaluation with the HUMAINE Framework
  5. 2025Measuring Political Preferences in AI Systems: An Integrative Approach
  6. 2024GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancy
  7. 2024Large Language Models (LLMs) as Agents for Augmented Democracy
  8. 2025From Delegates to Trustees: How Optimizing for Long-Term Interests Shapes Bias and Alignment in LLM