# Persona Non Grata: LLM Persona-Driven Generations in MCQA are Unstable in Distinct Dimensions

arXiv: https://arxiv.org/abs/2607.00937 · HTML: https://mlsanity.com/paper/2607.00937/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 1 Jul 2026.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models](https://mlsanity.com/paper/2305.18189/) (2023)
- [Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs](https://mlsanity.com/paper/2311.04892/) (2023)
- [Does Prompt Formatting Have Any Impact on LLM Performance?](https://mlsanity.com/paper/2411.10541/) (2024)
- [What's In My Big Data?](https://mlsanity.com/paper/2310.20707/) (2023)
- [Can LLM be a Personalized Judge?](https://mlsanity.com/paper/2406.11657/) (2024)
- [LLM Generated Persona is a Promise with a Catch](https://mlsanity.com/paper/2503.16527/) (2025)
- [Evaluating Large Language Model Biases in Persona-Steered Generation](https://mlsanity.com/paper/2405.20253/) (2024)
- [Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference](https://mlsanity.com/paper/2303.04673/) (2023)
- [Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems](https://mlsanity.com/paper/2310.05280/) (2023)
- [PatientSim: A Persona-Driven Simulator for Realistic Doctor-Patient Interactions](https://mlsanity.com/paper/2505.17818/) (2025)
- [Personalized Dialogue Generation with Persona-Adaptive Attention](https://mlsanity.com/paper/2210.15088/) (2022)
- [Leveraging Large Language Model as Simulated Patients for Clinical Education](https://mlsanity.com/paper/2404.13066/) (2024)
- [Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment](https://mlsanity.com/paper/2504.12663/) (2025)
- [A Large-Scale Simulation on Large Language Models for Decision-Making in Political Science](https://mlsanity.com/paper/2412.15291/) (2024)
- [MockLLM: A Multi-Agent Behavior Collaboration Framework for Online Job Seeking and Recruiting](https://mlsanity.com/paper/2405.18113/) (2024)
- [Simulating Human-Like Learning Dynamics with LLM-Empowered Agents](https://mlsanity.com/paper/2508.05622/) (2025)

## Abstract (arXiv)

Persona-driven generations (PDGs) have seen prolific use in research and industry applications, where a large language model (LLM) takes on a 'persona' while completing some task. While persona expressed through free-form text (like dialogue) has substantial work investigating stability or consistency, relatively, persona expressed in non-text-heavy outputs (like in multiple-choice question answering, or MCQA) is often overlooked. We work to address this gap, seeking to understand the instability of LLM PDGs in MCQA tasks. We develop three metrics investigating the performance, outcome, and question correctness stability, evaluating three distinct dimensions. Using these metrics, we find that instability varies consistently between model families and model size, and across question domains, with math/commonsense questions leading to greater instability. We also find task prompt format introduces more prediction instability than other hyperparameters, like temperature. Finally, we find that instability is related to task accuracy, and using our instability metrics, find different experimental settings that result in different best and worst personas for tasks, despite their similarity. This reveals the importance of checking hyperparameter instability in PDGs.