# Evaluating Large Language Model Biases in Persona-Steered Generation

arXiv: https://arxiv.org/abs/2405.20253 · HTML: https://mlsanity.com/paper/2405.20253/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 24 of 24)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [Toxicity in ChatGPT: Analyzing Persona-assigned Language Models](https://mlsanity.com/paper/2304.05335/) (2023)
- [Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2](https://mlsanity.com/paper/2311.10702/) (2023)
- [Do LLMs exhibit human-like response biases? A case study in survey design](https://mlsanity.com/paper/2311.04076/) (2023)
- [Quantifying the Persona Effect in LLM Simulations](https://mlsanity.com/paper/2402.10811/) (2024)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [Evaluating the Evaluation of Diversity in Natural Language Generation](https://mlsanity.com/paper/2004.02990/) (2020)
- [Aligning Language Models to User Opinions](https://mlsanity.com/paper/2305.14929/) (2023)
- [CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations](https://mlsanity.com/paper/2310.11501/) (2023)
- [Moral Foundations of Large Language Models](https://mlsanity.com/paper/2310.15337/) (2023)
- [Unintended Impacts of LLM Alignment on Global Representation](https://mlsanity.com/paper/2402.15018/) (2024)
- [Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems](https://mlsanity.com/paper/2310.05280/) (2023)
- [Semantic Diversity in Dialogue with Natural Language Inference](https://mlsanity.com/paper/2205.01497/) (2022)
- [HumBEL: A Human-in-the-Loop Approach for Evaluating Demographic Factors of Language Models in Human-Machine Conversations](https://mlsanity.com/paper/2305.14195/) (2023)

## Built on by (most cited first, 30 of 78)

- [PersonaGym: Evaluating Persona Agents and LLMs](https://mlsanity.com/paper/2407.18416/) (2024)
- [Virtual Personas for Language Models via an Anthology of Backstories](https://mlsanity.com/paper/2407.06576/) (2024)
- [The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Models](https://mlsanity.com/paper/2507.16076/) (2025)
- [This human study did not involve human subjects: Validating LLM simulations as behavioral evidence](https://mlsanity.com/paper/2602.15785/) (2026)
- [Exploring Safety-Utility Trade-Offs in Personalized Language Models](https://mlsanity.com/paper/2406.11107/) (2024)
- [Evaluating the Prompt Steerability of Large Language Models](https://mlsanity.com/paper/2411.12405/) (2024)
- [CoSER: A Comprehensive Literary Dataset and Framework for Training and Evaluating LLM Role-Playing and Persona Simulation](https://mlsanity.com/paper/2502.09082/) (2025)
- [Creativity in LLM-based Multi-Agent Systems: A Survey](https://mlsanity.com/paper/2505.21116/) (2025)
- [Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas](https://mlsanity.com/paper/2501.11549/) (2025)
- [Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics](https://mlsanity.com/paper/2506.12657/) (2025)
- [Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas](https://mlsanity.com/paper/2412.14843/) (2024)
- [Internal Value Alignment in Large Language Models through Controlled Value Vector Activation](https://mlsanity.com/paper/2507.11316/) (2025)
- [Scaffolding Empathy: Training Counselors with Simulated Patients and Utterance-level Performance Visualizations](https://mlsanity.com/paper/2502.18673/) (2025)
- [Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency](https://mlsanity.com/paper/2512.00461/) (2025)
- [LLM-Based Social Simulations Require a Boundary](https://mlsanity.com/paper/2506.19806/) (2025)
- [Exploring Changes in Nation Perception with Nationality-Assigned Personas in LLMs](https://mlsanity.com/paper/2406.13993/) (2024)
- [Are LLMs Empathetic to All? Investigating the Influence of Multi-Demographic Personas on a Model's Empathy](https://mlsanity.com/paper/2510.10328/) (2025)
- [Persona Setting Pitfall: Persistent Outgroup Biases in Large Language Models Arising from Social Identity Adoption](https://mlsanity.com/paper/2409.03843/) (2024)
- [Improving Language Model Personas via Rationalization with Psychological Scaffolds](https://mlsanity.com/paper/2504.17993/) (2025)
- [Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity](https://mlsanity.com/paper/2507.00657/) (2025)
- [Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation](https://mlsanity.com/paper/2506.19352/) (2025)
- [A Course Correction in Steerability Evaluation: Revealing Miscalibration and Side Effects in LLMs](https://mlsanity.com/paper/2505.23816/) (2025)
- [From Single to Societal: Analyzing Persona-Induced Bias in Multi-Agent Interactions](https://mlsanity.com/paper/2511.11789/) (2025)
- [Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essays](https://mlsanity.com/paper/2503.20062/) (2025)
- [Revealing Fine-Grained Values and Opinions in Large Language Models](https://mlsanity.com/paper/2406.19238/) (2024)
- [Political Ideology Shifts in Large Language Models](https://mlsanity.com/paper/2508.16013/) (2025)
- [To Mask or to Mirror: Human-AI Alignment in Collective Reasoning](https://mlsanity.com/paper/2510.01924/) (2025)
- [Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering](https://mlsanity.com/paper/2510.12925/) (2025)
- [Replicating Human Motivated Reasoning Studies with LLMs](https://mlsanity.com/paper/2601.16130/) (2026)
- [Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Tasks](https://mlsanity.com/paper/2509.09870/) (2025)

## Abstract (arXiv)

The task of persona-steered text generation requires large language models (LLMs) to generate text that reflects the distribution of views that an individual fitting a persona could have. People have multifaceted personas, but prior work on bias in LLM-generated opinions has only explored multiple-choice settings or one-dimensional personas. We define an incongruous persona as a persona with multiple traits where one trait makes its other traits less likely in human survey data, e.g. political liberals who support increased military spending. We find that LLMs are 9.7% less steerable towards incongruous personas than congruous ones, sometimes generating the stereotypical stance associated with its demographic rather than the target stance. Models that we evaluate that are fine-tuned with Reinforcement Learning from Human Feedback (RLHF) are more steerable, especially towards stances associated with political liberals and women, but present significantly less diverse views of personas. We also find variance in LLM steerability that cannot be predicted from multiple-choice opinion evaluation. Our results show the importance of evaluating models in open-ended text generation, as it can surface new LLM opinion biases. Moreover, such a setup can shed light on our ability to steer models toward a richer and more diverse range of viewpoints.