mlsanityData to 14 Oct 2026

From Demographics to Survey Anchors: Evaluating LLM Agents for Modeling Retirement Attitudes

arXiv 2605.16303 ↗First posted 24 Apr 2026cs.CY cs.AI cs.CLcited by 2

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Large language models (LLM) agents may offer tools to predict human responses to surveys. A common technique for defining these agents uses only demographics, for example country, age, gender, employment status, income, education and marital status. We compare the predictive accuracy of demographic agents to that of survey agents defined with a larger set of in-domain survey responses. We test both approaches in predicting responses to the multidisciplinary, cross-national Survey of Health, Ageing and Retirement in Europe (SHARE), focusing on five variables from three policy-relevant constructs around personal finance. In these three constructs, we observe that, compared to survey agents trained on broader data, demographics-only agents (1) exhibited a central tendency bias, skewing answers toward population means, and (2) were unrealistically accurate, failing to reproduce the incorrect answers and "don't know" responses typical of human respondents. These performance differences are further substantiated through the replication of a hierarchical regression analysis from prior retirement planning research. Agents based solely on demographic information reproduce the outcome that financial risk tolerance, future time perspective, and knowledge of retirement planning each are predictive of retirement savings. However, only the survey-anchored agents succeed in reproducing the interaction among these three factors. These findings suggest caution in using only demographics to define LLM agents for predicting survey responses.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 28 of its 89 references that are in the map. First posted on arXiv 24 Apr 2026.

Builds on

28 of 89 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 27

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2025Gemma 3 Technical Reportcited by 1,824
  6. 2023Whose Opinions Do Language Models Reflect?cited by 773
  7. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  8. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  9. 2022Using cognitive psychology to understand GPT-3cited by 398
  10. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
  11. 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 144
  12. 2025LLM Social Simulations Are a Promising Research Methodcited by 99
15 more
  1. 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 66
  2. 2023The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspectivecited by 59
  3. 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversitycited by 58
  4. 2023AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Predictioncited by 48
  5. 2023Diminished Diversity-of-Thought in a Standard Large Language Modelcited by 45
  6. 2023TRAM: Benchmarking Temporal Reasoning for Large Language Modelscited by 40
  7. 2024The illusion of artificial inclusioncited by 34
  8. 2023Language Models Trained on Media Diets Can Predict Public Opinioncited by 30
  9. 2024'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participantscited by 28
  10. 2025Twin-2K-500: A dataset for building digital twins of over 2,000 people based on their answers to over 500 questionscited by 22
  11. 2025Classroom Simulacra: Building Contextual Student Generative Agents in Online Education for Learning Behavioral Simulationcited by 15
  12. 2024Time Awareness in Large Language Models: Benchmarking Fact Recall Across Timecited by 13
  13. 2024Temporal Blind Spots in Large Language Modelscited by 7
  14. 2025Large Language Models as Virtual Survey Respondents: Evaluating Sociodemographic Response Generationcited by 7
  15. 2025SocioBench: Modeling Human Behavior in Sociological Surveys with Large Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Behavioral History Outperforms Descriptions of the Person for LLM Synthetic Personascited by 0
  2. 2026HouseholdBench: Evaluating Large Language Models as Predictors of Household Economic Behaviorcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluating the Bias in LLMs for Surveying Opinion and Decision Making in Healthcare
  2. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals
  3. 2025Simulating Macroeconomic Expectations in Survey Experiments with LLM-based Economic Agents
  4. 2025Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction
  5. 2025An Analysis of Large Language Models for Simulating User Responses in Surveys
  6. 2026From Survey Personas to LLM Agents: A Generative Agent-based Simulation of Mobility Policy Preference Dynamics
  7. 2026Can Large Language Models Anticipate Behavioral Responses to Social Policies? A Case of Pension Enrollment Prediction among China's Flexible Workers
  8. 2026Beyond Marginal Distributions: A Framework to Evaluate the Representativeness of Demographic-Aligned LLMs