mlsanityData to 7 Oct 2026

Evaluating Large Language Model Biases in Persona-Steered Generation

arXiv 2405.20253 ↗First posted 30 May 2024cs.CLcited by 78

Not on the map. Why?

Abstract (arXiv)

The task of persona-steered text generation requires large language models (LLMs) to generate text that reflects the distribution of views that an individual fitting a persona could have. People have multifaceted personas, but prior work on bias in LLM-generated opinions has only explored multiple-choice settings or one-dimensional personas. We define an incongruous persona as a persona with multiple traits where one trait makes its other traits less likely in human survey data, e.g. political liberals who support increased military spending. We find that LLMs are 9.7% less steerable towards incongruous personas than congruous ones, sometimes generating the stereotypical stance associated with its demographic rather than the target stance. Models that we evaluate that are fine-tuned with Reinforcement Learning from Human Feedback (RLHF) are more steerable, especially towards stances associated with political liberals and women, but present significantly less diverse views of personas. We also find variance in LLM steerability that cannot be predicted from multiple-choice opinion evaluation. Our results show the importance of evaluating models in open-ended text generation, as it can surface new LLM opinion biases. Moreover, such a setup can shed light on our ability to steer models toward a richer and more diverse range of viewpoints.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

24 of 38 references.

References · 24

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  5. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  6. 2023Whose Opinions Do Language Models Reflect?cited by 759
  7. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  8. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  9. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  10. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  11. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  12. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
12 more
  1. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 227
  2. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 176
  3. 2024Quantifying the Persona Effect in LLM Simulationscited by 164
  4. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  5. 2020Evaluating the Evaluation of Diversity in Natural Language Generationcited by 127
  6. 2023Aligning Language Models to User Opinionscited by 98
  7. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 94
  8. 2023Moral Foundations of Large Language Modelscited by 84
  9. 2024Unintended Impacts of LLM Alignment on Global Representationcited by 66
  10. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
  11. 2022Semantic Diversity in Dialogue with Natural Language Inferencecited by 28
  12. 2023HumBEL: A Human-in-the-Loop Approach for Evaluating Demographic Factors of Language Models in Human-Machine Conversationscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 42; 2026: 22.

78 papers.

Sort:

By area:

  1. 2024PersonaGym: Evaluating Persona Agents and LLMscited by 89
  2. 2024Virtual Personas for Language Models via an Anthology of Backstoriescited by 49
  3. 2025The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Modelscited by 40
  4. 2024Exploring Safety-Utility Trade-Offs in Personalized Language Modelscited by 26
  5. 2024Evaluating the Prompt Steerability of Large Language Modelscited by 26
  6. 2025CoSER: A Comprehensive Literary Dataset and Framework for Training and Evaluating LLM Role-Playing and Persona Simulationcited by 26
  7. 2026This human study did not involve human subjects: Validating LLM simulations as behavioral evidencecited by 26
  8. 2025Creativity in LLM-based Multi-Agent Systems: A Surveycited by 23
  9. 2025Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamicscited by 19
  10. 2025Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personascited by 17
  11. 2024Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personascited by 16
  12. 2025Scaffolding Empathy: Training Counselors with Simulated Patients and Utterance-level Performance Visualizationscited by 14
38 more
  1. 2025Internal Value Alignment in Large Language Models through Controlled Value Vector Activationcited by 14
  2. 2025Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparencycited by 14
  3. 2024Exploring Changes in Nation Perception with Nationality-Assigned Personas in LLMscited by 11
  4. 2025LLM-Based Social Simulations Require a Boundarycited by 11
  5. 2024Persona Setting Pitfall: Persistent Outgroup Biases in Large Language Models Arising from Social Identity Adoptioncited by 9
  6. 2025Improving Language Model Personas via Rationalization with Psychological Scaffoldscited by 9
  7. 2025Are LLMs Empathetic to All? Investigating the Influence of Multi-Demographic Personas on a Model's Empathycited by 9
  8. 2025Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generationcited by 8
  9. 2025Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicitycited by 8
  10. 2025A Course Correction in Steerability Evaluation: Revealing Miscalibration and Side Effects in LLMscited by 7
  11. 2025From Single to Societal: Analyzing Persona-Induced Bias in Multi-Agent Interactionscited by 7
  12. 2025Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essayscited by 6
  13. 2024Revealing Fine-Grained Values and Opinions in Large Language Modelscited by 5
  14. 2025Political Ideology Shifts in Large Language Modelscited by 5
  15. 2025To Mask or to Mirror: Human-AI Alignment in Collective Reasoningcited by 5
  16. 2026Replicating Human Motivated Reasoning Studies with LLMscited by 5
  17. 2025Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Taskscited by 4
  18. 2025Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answeringcited by 4
  19. 2024SubData: Bridging Heterogeneous Datasets to Enable Theory-Driven Evaluation of Political and Demographic Perspectives in LLMscited by 3
  20. 2025Ontologies in Design: How Imagining a Tree Reveals Possibilites and Assumptions in Large Language Modelscited by 3
  21. 2025Localizing Persona Representations in LLMscited by 3
  22. 2025Are Economists Always More Introverted? Analyzing Consistency in Persona-Assigned LLMscited by 3
  23. 2025EVALUESTEER: Measuring Reward Model Steerability Towards Values and Preferencescited by 3
  24. 2025Simulating and Experimenting with Social Media Mobilization Using LLM Agentscited by 3
  25. 2026Persona Prompting as a Lens on LLM Social Reasoningcited by 3
  26. 2024Inertia in Moral and Value Judgments of Large Language Modelscited by 2
  27. 2024In-Context Learning for Preserving Patient Privacy: A Framework for Synthesizing Realistic Patient Portal Messagescited by 2
  28. 2025Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detectioncited by 2
  29. 2025Persona Alchemy: Designing, Evaluating, and Implementing Psychologically-Grounded LLM Agents for Diverse Stakeholder Representationcited by 2
  30. 2025From Anger to Joy: How Nationality Personas Shape Emotion Attribution in Large Language Modelscited by 2
  31. 2025Simulating Identity, Propagating Bias: Abstraction and Stereotypes in LLM-Generated Textcited by 2
  32. 2025Human Psychometric Questionnaires Mischaracterize LLM Behaviorcited by 2
  33. 2026Tastes without distinction: silicon samples and the synthetic construction of tastescited by 2
  34. 2026Structuring the Space of Sociotechnical Alignmentcited by 2
  35. 2024Scenarios and Approaches for Situated Natural Language Explanationscited by 1
  36. 2024Prompt and Prejudicecited by 1
  37. 2025Biases in Large Language Model-Elicited Text: A Case Study in Natural Language Inferencecited by 1
  38. 2025Hateful Person or Hateful Model? Investigating the Role of Personas in Hate Speech Detection by Large Language Modelscited by 1

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Aligning Large Language Models with Human Opinions through Persona Selection and Value--Belief--Norm Reasoning
  2. 2023Intentional Biases in LLM Responses
  3. 2026Persona-Based Simulation of Human Opinion at Population Scale
  4. 2024Political Bias in LLMs: Unaligned Moral Values in Agent-centric Simulations
  5. 2024PERSONA: A Reproducible Testbed for Pluralistic Alignment
  6. 2026Multi-Persona Thinking for Bias Mitigation in Large Language Models
  7. 2026Invisible Influences: Investigating Implicit Intersectional Biases through Persona Engineering in Large Language Models
  8. 2025Us-vs-Them bias in Large Language Models ↗