mlsanityData to 7 Oct 2026

Evaluating and Inducing Personality in Pre-trained Language Models

arXiv 2206.07550 ↗First posted 20 May 2022cs.CL cs.AI cs.LGcited by 122

Not on the map. Why?

Cited by 26 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Standardized and quantified evaluation of machine behaviors is a crux of understanding LLMs. In this study, we draw inspiration from psychometric studies by leveraging human personality theory as a tool for studying machine behaviors. Originating as a philosophical quest for human behaviors, the study of personality delves into how individuals differ in thinking, feeling, and behaving. Toward building and understanding human-like social machines, we are motivated to ask: Can we assess machine behaviors by leveraging human psychometric tests in a principled and quantitative manner? If so, can we induce a specific personality in LLMs? To answer these questions, we introduce the Machine Personality Inventory (MPI) tool for studying machine behaviors; MPI follows standardized personality tests, built upon the Big Five Personality Factors (Big Five) theory and personality assessment inventories. By systematically evaluating LLMs with MPI, we provide the first piece of evidence demonstrating the efficacy of MPI in studying LLMs behaviors. We further devise a Personality Prompting (P^2) method to induce LLMs with specific personalities in a controllable way, capable of producing diverse and verifiable behaviors. We hope this work sheds light on future studies by adopting personality as the essential indicator for various downstream tasks, and could further motivate research into equally intriguing human-like machine behaviors.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). Builds on 31 of its 84 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

31 of 84 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  11. 2022Red Teaming Language Models with Language Modelscited by 983
  12. 2019How Can We Know What Language Models Know?cited by 953
19 more
  1. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  2. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  3. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  4. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  5. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  6. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
  7. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  8. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  9. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
  10. 2018Measuring abstract reasoning in neural networkscited by 165
  11. 2021Understanding the Capabilities, Limitations, and Societal Impact of Large Language Modelscited by 159
  12. 2021OpenPrompt: An Open-source Framework for Prompt-learningcited by 148
  13. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  14. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  15. 2020Few-Shot Text Generation with Pattern-Exploiting Trainingcited by 87
  16. 2020Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sensecited by 62
  17. 2020How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Contextcited by 53
  18. 2021Structural Adapters in Pretrained Language Models for AMR-to-text Generationcited by 48
  19. 2020Technical Report: Auxiliary Tuning and its Application to Conditional Text Generationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 34; 2025: 37; 2026: 39.

122 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023Challenges and Applications of Large Language Modelscited by 251
  3. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  4. 2024A Roadmap to Pluralistic Alignmentcited by 175
  5. 2023Personality Traits in Large Language Modelscited by 166
  6. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  7. 2024Aligning to Thousands of Preferences via System Message Generalizationcited by 84
  8. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  9. 2023Turning large language models into cognitive modelscited by 66
  10. 2023Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBenchcited by 54
  11. 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 49
  12. 2025Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactionscited by 47
38 more
  1. 2024Y Social: an LLM-powered Social Media Digital Twincited by 43
  2. 2024Personality Alignment of Large Language Modelscited by 36
  3. 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMscited by 35
  4. 2025LoRe: Personalizing LLMs via Low-Rank Reward Modelingcited by 34
  5. 2024From Distributional to Overton Pluralism: Investigating Large Language Model Alignmentcited by 33
  6. 2023Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBenchcited by 28
  7. 2024PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learningcited by 28
  8. 2023LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMscited by 27
  9. 2025A Survey of Personalization: From RAG to Agentcited by 26
  10. 2023Challenging the Validity of Personality Tests for Large Language Modelscited by 25
  11. 2025Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation Historycited by 25
  12. 2025Evaluating Personality Traits in Large Language Models: Insights from Psychological Questionnairescited by 24
  13. 2024A Study on Large Language Models' Limitations in Multiple-Choice Question Answeringcited by 22
  14. 2024Evaluating Large Language Models with Psychometricscited by 22
  15. 2024Helpful assistant or fruitful facilitator? Investigating how personas affect language model behaviorcited by 21
  16. 2024The Better Angels of Machine Personality: How Personality Relates to LLM Safetycited by 21
  17. 2023Revisiting the Reliability of Psychological Scales on Large Language Modelscited by 16
  18. 2024MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needscited by 16
  19. 2023On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlookcited by 15
  20. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
  21. 2024Stick to your Role! Stability of Personal Values Expressed in Large Language Modelscited by 14
  22. 2024From Imitation to Introspection: Probing Self-Consciousness in Language Modelscited by 14
  23. 2024Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approachescited by 13
  24. 2025Ready Jurist One: Benchmarking Language Agents for Legal Intelligence in Dynamic Environmentscited by 13
  25. 2025The Era of Real-World Human Interaction: RL from User Conversationscited by 13
  26. 2024Spontaneous Emergence of Agent Individuality through Social Interactions in LLM-Based Communitiescited by 12
  27. 2025MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systemscited by 12
  28. 2025Pairwise Calibrated Rewards for Pluralistic Alignmentcited by 12
  29. 2026PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebracited by 11
  30. 2024Neuron-based Personality Trait Induction in Large Language Modelscited by 9
  31. 2025Probing then Editing Response Personality of Large Language Modelscited by 9
  32. 2025Value Drifts: Tracing Value Alignment During LLM Post-Trainingcited by 9
  33. 2024Effective Generative AI: The Human-Algorithm Centaurcited by 8
  34. 2024Assessing Social Alignment: Do Personality-Prompted Large Language Models Behave Like Humans?cited by 8
  35. 2025How Personality Traits Shape LLM Risk-Taking Behaviourcited by 8
  36. 2025Bullying the Machine: How Personas Increase LLM Vulnerabilitycited by 8
  37. 2024AFSPP: Agent Framework for Shaping Preference and Personality with Large Language Modelscited by 7
  38. 2024Large Language Models Can Infer Personality from Free-Form User Interactionscited by 7

50 most cited of 122 All 122 (JSON)