# Are Economists Always More Introverted? Analyzing Consistency in Persona-Assigned LLMs

arXiv: https://arxiv.org/abs/2506.02659 · HTML: https://mlsanity.com/paper/2506.02659/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 20 of 20)

- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [Character-LLM: A Trainable Agent for Role-Playing](https://mlsanity.com/paper/2310.10158/) (2023)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs](https://mlsanity.com/paper/2311.04892/) (2023)
- [InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews](https://mlsanity.com/paper/2310.17976/) (2023)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [Evaluating Large Language Model Biases in Persona-Steered Generation](https://mlsanity.com/paper/2405.20253/) (2024)
- [Calibrating Large Language Models with Sample Consistency](https://mlsanity.com/paper/2402.13904/) (2024)
- [Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics](https://mlsanity.com/paper/2406.14703/) (2024)
- [Large Language Models Reflect the Ideology of their Creators](https://mlsanity.com/paper/2410.18417/) (2024)
- [On the steerability of large language models toward data-driven personas](https://mlsanity.com/paper/2311.04978/) (2023)
- [You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments](https://mlsanity.com/paper/2311.09718/) (2023)
- [The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions](https://mlsanity.com/paper/2310.12418/) (2023)
- [Model Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science](https://mlsanity.com/paper/2203.07031/) (2022)
- [To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity](https://mlsanity.com/paper/2407.17125/) (2024)
- [Robustness and Confounders in the Demographic Alignment of LLMs with Human Perceptions of Offensiveness](https://mlsanity.com/paper/2411.08977/) (2024)

## Built on by (most cited first, 3 of 3)

- [Personality Expression Across Contexts: Linguistic and Behavioral Variation in LLM Agents](https://mlsanity.com/paper/2602.01063/) (2026)
- [Would a Large Language Model Pay Extra for a View? Inferring Willingness to Pay from Subjective Choices](https://mlsanity.com/paper/2602.09802/) (2026)
- [LLM Consumer Behavior Theory: Foundations of a Novel Research Field](https://mlsanity.com/paper/2606.18005/) (2026)

## Abstract (arXiv)

Personalized Large Language Models (LLMs) are increasingly used in diverse applications, where they are assigned a specific persona - such as a happy high school teacher - to guide their responses. While prior research has examined how well LLMs adhere to predefined personas in writing style, a comprehensive analysis of consistency across different personas and task types is lacking. In this paper, we introduce a new standardized framework to analyze consistency in persona-assigned LLMs. We define consistency as the extent to which a model maintains coherent responses when assigned the same persona across different tasks and runs. Our framework evaluates personas across four different categories (happiness, occupation, personality, and political stance) spanning multiple task dimensions (survey writing, essay generation, social media post generation, single turn, and multi-turn conversations). Our findings reveal that consistency is influenced by multiple factors, including the assigned persona, stereotypes, and model design choices. Consistency also varies across tasks, increasing with more structured tasks and additional context. All code is available on GitHub.