mlsanityData to 7 Oct 2026

Distributional Validity of a Korean Synthetic Persona Panel: Evidence From the Korea Media Panel Survey

arXiv 2608.28615 ↗First posted 23 Jul 2026cs.CY cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) personas are proposed as survey respondents, yet validation outside English-speaking contexts is scarce. We evaluate how well a Korean synthetic persona panel used to condition Gemini 3.5 Flash and EXAONE reproduces digital and artificial intelligence (AI) service-use distributions of the Korea Media Panel Survey. About 8,000 personas per model answered eight service-use items and eight attitudinal constructs; responses were compared with weighted survey estimates. The overall mean absolute error (MAE) was 14-19 percentage points (pp), with binary item-mean correlations of 0.70-0.91 across waves. Segment error across five axes was 14-18 pp, with between-group signed-error ranges of 49.6/34.7 pp (Gemini/EXAONE; 39.5/31.2 without the non-comparable teen cells). Errors were model-specific: an age stereotype (Gemini) versus an acquiescence-consistent level bias (EXAONE). Generative-AI overestimation was consistent with temporal misalignment; short-form underestimation was framing-sensitive and persisted under randomized order (both shown for Gemini). Post-hoc holdout calibration on 30% of the real data, with the correction form selected inside the calibration set, cut cell MAE from 18.3/15.1 to 4.9/4.4 pp, yet direct estimation from that subsample was more accurate than the calibrated panel (3.6 pp), a synthetic-informed shrinkage estimator beat its real-only counterpart by at most 0.7 pp, and the correction did not transfer competitively across waves. The calibrated panel kept an advantage only below roughly 250-860 real responses (at most 2.3 pp over a real-only shrinkage estimator) or, for one model, on unobserved segments. In this setting, synthetic panels are not survey substitutes; their value is diagnostic.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

17 of 31 references.

References · 17

  1. 2023Whose Opinions Do Language Models Reflect?cited by 759
  2. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  3. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  4. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 235
  5. 2023Questioning the Survey Responses of Large Language Modelscited by 70
  6. 2025Specializing Large Language Models to Simulate Survey Response Distributions for Global Populationscited by 50
  7. 2023Diminished Diversity-of-Thought in a Standard Large Language Modelcited by 43
  8. 2024Large Language Models for Market Research: A Data-augmentation Approachcited by 18
  9. 2025Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparencycited by 14
  10. 2025Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Modelscited by 8
  11. 2026Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondentscited by 7
  12. 2025Acquiescence Bias in Large Language Modelscited by 5
5 more
  1. 2026Plausible but Not Valid: A Psychometric Audit of LLMs as Synthetic Survey Respondentscited by 4
  2. 2025Polypersona: Persona-Grounded LLM for Synthetic Survey Responsescited by 3
  3. 2026When Can Digital Personas Reliably Approximate Human Survey Findings?cited by 2
  4. 2025Emulating Public Opinion: A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Casecited by 1
  5. 2026Improving Cross-Cultural Survey Simulation with Calibrated Value Personascited by 1

Built on by

Not cited yet.