mlsanityData to 7 Oct 2026

PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits

arXiv 2305.02547 ↗First posted 4 May 2023cs.CL cs.AI cs.HCcited by 229

Not on the map. Why?

Cited by 55 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite the many use cases for large language models (LLMs) in creating personalized chatbots, there has been limited research on evaluating the extent to which the behaviors of personalized LLMs accurately and consistently reflect specific personality traits. We consider studying the behavior of LLM-based agents which we refer to as LLM personas and present a case study with GPT-3.5 and GPT-4 to investigate whether LLMs can generate content that aligns with their assigned personality profiles. To this end, we simulate distinct LLM personas based on the Big Five personality model, have them complete the 44-item Big Five Inventory (BFI) personality test and a story writing task, and then assess their essays with automatic and human evaluations. Results show that LLM personas' self-reported BFI scores are consistent with their designated personality types, with large effect sizes observed across five traits. Additionally, LLM personas' writings have emerging representative linguistic patterns for personality traits when compared with a human writing corpus. Furthermore, human evaluation shows that humans can perceive some personality traits with an accuracy of up to 80%. Interestingly, the accuracy drops significantly when the annotators were informed of AI authorship.

Key facts. Cited by 229 papers in the map (data through 7 Oct 2026). Builds on 12 of its 60 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

12 of 60 references.

References · 12

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  3. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  4. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  5. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  6. 2022Using cognitive psychology to understand GPT-3cited by 389
  7. 2022Language models show human-like content effects on reasoning taskscited by 171
  8. 2020Politeness Transfer: A Tag and Generate Approachcited by 78
  9. 2023Can ChatGPT Assess Human Personalities? A General Evaluation Frameworkcited by 69
  10. 2022Estimating the Personality of White-Box Language Modelscited by 47
  11. 2022Identifying and Manipulating the Personality Traits of Language Modelscited by 40
  12. 2019IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translationcited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 24; 2025: 104; 2026: 98.

229 papers.

Sort:

By area:

  1. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  2. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 163
  3. 2025LLM Social Simulations Are a Promising Research Methodcited by 97
  4. 2024Aligning to Thousands of Preferences via System Message Generalizationcited by 84
  5. 2025Voting or Consensus? Decision-Making in Multi-Agent Debatecited by 55
  6. 2023Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBenchcited by 54
  7. 2024Virtual Personas for Language Models via an Anthology of Backstoriescited by 49
  8. 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 49
  9. 2023CGMI: Configurable General Multi-Agent Interaction Frameworkcited by 41
  10. 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMscited by 35
  11. 2025A Closer Look at System Prompt Robustnesscited by 31
  12. 2025Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learningcited by 27
38 more
  1. 2025SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMscited by 26
  2. 2024TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Studentscited by 25
  3. 2025Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation Historycited by 25
  4. 2025Evaluating Personality Traits in Large Language Models: Insights from Psychological Questionnairescited by 24
  5. 2025Mechanistic Interpretability of Emotion Inference in Large Language Modelscited by 24
  6. 2024Towards a Client-Centered Assessment of LLM Therapists by Client Simulationcited by 23
  7. 2025From Personas to Talks: Revisiting the Impact of Personas on LLM-Synthesized Emotional Support Conversationscited by 23
  8. 2025Creativity in LLM-based Multi-Agent Systems: A Surveycited by 23
  9. 2024Evaluating Large Language Models with Psychometricscited by 22
  10. 2025TrajLLM: A Modular LLM-Enhanced Agent-Based Framework for Realistic Human Trajectory Simulationcited by 21
  11. 2024The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Modelscited by 19
  12. 2025Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMscited by 19
  13. 2025Are Generative AI Agents Effective Personalized Financial Advisors?cited by 18
  14. 2024Plurals: A System for Guiding LLMs Via Simulated Social Ensemblescited by 17
  15. 2025Personality Vector: Modulating Personality of Large Language Models by Model Mergingcited by 16
  16. 2025Can LLM Agents Maintain a Persona in Discourse?cited by 15
  17. 2025Biased Tales: Cultural and Topic Bias in Generating Children's Storiescited by 15
  18. 2025Exploring the Impact of Personality Traits on LLM Bias and Toxicitycited by 14
  19. 2025Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparencycited by 14
  20. 2026A Survey on LLM-based Conversational User Simulationcited by 14
  21. 2024Large language models can consistently generate high-quality content for election disinformation operationscited by 13
  22. 2024Measuring Human and AI Values Based on Generative Psychometrics with Large Language Modelscited by 13
  23. 2024Identifying and Manipulating Personality Traits in LLMs Through Activation Engineeringcited by 13
  24. 2024A Character-Centric Creative Story Generation via Imaginationcited by 12
  25. 2024How Does the Disclosure of AI Assistance Affect the Perceptions of Writing?cited by 12
  26. 2024Building Knowledge-Guided Lexica to Model Cultural Variationcited by 11
  27. 2026PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebracited by 11
  28. 2024Self-assessment, Exhibition, and Recognition: a Review of Personality in Large Language Modelscited by 10
  29. 2025Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applicationscited by 10
  30. 2024Step-Back Profiling: Distilling User History for Personalized Scientific Writingcited by 9
  31. 2025Probing then Editing Response Personality of Large Language Modelscited by 9
  32. 2025Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignmentcited by 9
  33. 2025From Prompts to Constructs: A Dual-Validity Framework for Large Language Model Research in Psychologycited by 9
  34. 2024MirrorStories: Reflecting Diversity through Personalized Narrative Generation with Large Language Modelscited by 8
  35. 2025SPeCtrum: A Grounded Framework for Multidimensional Identity Representation in LLM-Based Agentcited by 8
  36. 2025The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agentscited by 8
  37. 2025Bullying the Machine: How Personas Increase LLM Vulnerabilitycited by 8
  38. 2025Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generationcited by 8

50 most cited of 229 All 229 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models
  2. 2024P-React: Synthesizing Topic-Adaptive Reactions of Personality Traits via Mixture of Specialized LoRA Experts
  3. 2025Large Language Model Agent Personality and Response Appropriateness: Evaluation by Human Linguistic Experts, LLM-as-Judge, and Natural Language Processing Model
  4. 2024Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models
  5. 2024Is persona enough for personality? Using ChatGPT to reconstruct an agent's latent personality from simple descriptions
  6. 2025Beyond BFI: The CSI for Enhanced Reliability and Validity in Evaluating LLM Personality Traits
  7. 2025Can LLMs Infer Personality from Real World Conversations?
  8. 2026Do Personality-Tuned LLMs Make Better Social Agents?