mlsanityData to 14 Oct 2026

Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personas

arXiv 2408.06929 ↗First posted 13 Aug 2024cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

The success of Large Language Models (LLMs) in multicultural environments hinges on their ability to understand users' diverse cultural backgrounds. We measure this capability by having an LLM simulate human profiles representing various nationalities within the scope of a questionnaire-style psychological experiment. Specifically, we employ GPT-3.5 to reproduce reactions to persuasive news articles of 7,286 participants from 15 countries; comparing the results with a dataset of real participants sharing the same demographic traits. Our analysis shows that specifying a person's country of residence improves GPT-3.5's alignment with their responses. In contrast, using native language prompting introduces shifts that significantly reduce overall alignment, with some languages particularly impairing performance. These findings suggest that while direct nationality information enhances the model's cultural adaptability, native language cues do not reliably improve simulation fidelity and can detract from the model's effectiveness.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Builds on 40 of its 68 references that are in the map. First posted on arXiv 13 Aug 2024.

Builds on

40 of 68 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2023Whose Opinions Do Language Models Reflect?cited by 773
  7. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  8. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  9. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  10. 2023Role-Play with Large Language Modelscited by 294
  11. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  12. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 206
28 more
  1. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 192
  2. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 189
  3. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  4. 2023Personality Traits in Large Language Modelscited by 169
  5. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  6. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 149
  7. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 120
  8. 2022Who is GPT-3? An Exploration of Personality, Values and Demographicscited by 112
  9. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 104
  10. 2023Aligning Language Models to User Opinionscited by 99
  11. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 90
  12. 2024Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuningcited by 90
  13. 2023Moral Foundations of Large Language Modelscited by 85
  14. 2023Questioning the Survey Responses of Large Language Modelscited by 74
  15. 2023Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithmscited by 65
  16. 2023Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Modelscited by 56
  17. 2023Multilingual Language Models are not Multicultural: A Case Study in Emotioncited by 54
  18. 2023Do LLM Agents Exhibit Social Behavior?cited by 53
  19. 2024Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Informationcited by 49
  20. 2024Dialect prejudice predicts AI decisions about people's character, employability, and criminalitycited by 37
  21. 2024Gender Bias in Large Language Models across Multiple Languagescited by 35
  22. 2023Language Models Trained on Media Diets Can Predict Public Opinioncited by 30
  23. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  24. 2021Quantifying Gender Bias Towards Politicians in Cross-Lingual Language Modelscited by 15
  25. 2021An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregiverscited by 15
  26. 2023Multilingual large language models leak human stereotypes across language boundariescited by 12
  27. 2024English Prompts are Better for NLI-based Zero-Shot Emotion Classification than Target-Language Promptscited by 7
  28. 2024Global-Liar: Factuality of LLMs over Time and Geographic Regionscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 23; 2026: 13.

37 papers.

Sort:

By area:

  1. 2025Specializing Large Language Models to Simulate Survey Response Distributions for Global Populationscited by 53
  2. 2025The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Modelscited by 42
  3. 2025LoRe: Personalizing LLMs via Low-Rank Reward Modelingcited by 38
  4. 2024LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Outputcited by 27
  5. 2025Multilingual Prompting for Improving LLM Generation Diversitycited by 25
  6. 2025Large Language Models Do Not Simulate Human Psychologycited by 19
  7. 2025SOTOPIA-S4: a user-friendly system for flexible, customizable, and large-scale social simulationcited by 10
  8. 2025Are LLMs Empathetic to All? Investigating the Influence of Multi-Demographic Personas on a Model's Empathycited by 10
  9. 2025SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabiacited by 8
  10. 2025CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimizationcited by 6
  11. 2025Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Modelscited by 5
  12. 2025Made-in China, Thinking in America:U.S. Values Persist in Chinese LLMscited by 5
25 more
  1. 2025AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Differencecited by 4
  2. 2025Beyond Demographics: Enhancing Cultural Value Survey Simulation with Multi-Stage Personality-Driven Cognitive Reasoningcited by 4
  3. 2025Rethinking AI Cultural Alignmentcited by 3
  4. 2025Exploring Persona Sentiment Sensitivity in Personalized Dialogue Generationcited by 3
  5. 2025Improving the Distributional Alignment of LLMs using Supervisioncited by 2
  6. 2026Stories of Your Life as Others: A Round-Trip Evaluation of LLM-Generated Life Stories Conditioned on Rich Psychometric Profilescited by 2
  7. 2026Lessons Without Borders? Evaluating Cultural Alignment of LLMs Using Multilingual Story Moral Generationcited by 2
  8. 2025Free Lunch for User Experience: Crowdsourcing Agents for Scalable User Studiescited by 1
  9. 2025The American Ghost in the Machine: How language models align culturally and the effects of cultural promptingcited by 1
  10. 2026Prompt Programming for Cultural Bias and Alignment of Large Language Modelscited by 1
  11. 2026Beyond Averages: Evaluating LLMs on Human Survey Replication at the Distributional Levelcited by 1
  12. 2025Assessing Agentic Large Language Models in Multilingual National Biascited by 0
  13. 2025Visual moral inference and communicationcited by 0
  14. 2025Images Amplify Misinformation Sharing in Vision-Language Modelscited by 0
  15. 2025Measure what Matters: Psychometric Evaluation of AI with Situational Judgment Testscited by 0
  16. 2025DeepSeek's WEIRD Behavior: The cultural alignment of Large Language Models and the effects of prompt language and cultural promptingcited by 0
  17. 2026From National Curricula to Cultural Awareness: Constructing Open-Ended Culture-Specific Question Answering Datasetcited by 0
  18. 2026Occupational Prompting Reveals Cultural Bias in Large Language Modelscited by 0
  19. 2026Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communitiescited by 0
  20. 2026NRITYAM: Language Models Meet Art and Heritage of Dancecited by 0
  21. 2026AI YOU Town: Make Friends and Money with Your Digital Twincited by 0
  22. 2026ExpertIVS: Sociological Expert Driven Individual Value Simulation in Large Language Modelscited by 0
  23. 2026DiSCo: A Distribution-First Steering and Cultural Prior Evaluation Framework for Measuring Cultural Preference Bias in LLMscited by 0
  24. 2026Reach Into The CHOIR: Free-List Elicitation Uncovers Distinct Model Voices in LLM Ensemblescited by 0
  25. 2026Data-Driven Personas for Survey Simulation: Insights into Simulation Alignment Across Data-Access Regimescited by 0