# LMLPA: Language Model Linguistic Personality Assessment

arXiv: https://arxiv.org/abs/2410.17632 · HTML: https://mlsanity.com/paper/2410.17632/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 98 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 13 of 13)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://mlsanity.com/paper/1909.00161/) (2019)
- [Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions](https://mlsanity.com/paper/2308.11483/) (2023)
- [Who is GPT-3? An Exploration of Personality, Values and Demographics](https://mlsanity.com/paper/2209.14338/) (2022)
- [Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models](https://mlsanity.com/paper/2307.16180/) (2023)
- [Estimating the Personality of White-Box Language Models](https://mlsanity.com/paper/2204.12000/) (2022)
- [Identifying and Manipulating the Personality Traits of Language Models](https://mlsanity.com/paper/2212.10276/) (2022)
- [The Effects of Embodiment and Personality Expression on Learning in LLM-based Educational Agents](https://mlsanity.com/paper/2407.10993/) (2024)

## Built on by (most cited first, 15 of 15)

- [Value Portrait: Assessing Language Models' Values through Psychometrically and Ecologically Valid Items](https://mlsanity.com/paper/2505.01015/) (2025)
- [Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications](https://mlsanity.com/paper/2505.00049/) (2025)
- [Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations](https://mlsanity.com/paper/2601.09833/) (2026)
- [AI-exhibited Personality Traits Can Shape Human Self-concept through Conversations](https://mlsanity.com/paper/2601.12727/) (2026)
- [Integrating Personality into Digital Humans: A Review of LLM-Driven Approaches for Virtual Reality](https://mlsanity.com/paper/2503.16457/) (2025)
- [Style over Story: Measuring LLM Narrative Preferences via Structured Selection](https://mlsanity.com/paper/2510.02025/) (2025)
- [Quantifying and Mitigating Socially Desirable Responding in LLMs: A Desirability-Matched Graded Forced-Choice Psychometric Study](https://mlsanity.com/paper/2602.17262/) (2026)
- [Enhancing Zero-shot Personalized Image Aesthetics Assessment with Profile-aware Multimodal LLM](https://mlsanity.com/paper/2604.17233/) (2026)
- [When AI Takes the Couch: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models](https://mlsanity.com/paper/2512.04124/) (2025)
- [Personality Shapes Gender Bias in Persona-Conditioned LLM Narratives Across English and Hindi: An Empirical Investigation](https://mlsanity.com/paper/2604.23600/) (2026)
- [Narrative Landscape: Mapping Narrative Dispositions Across LLMs](https://mlsanity.com/paper/2605.08742/) (2026)
- [The Unsampled Truth: Quantifying Prompt Artifacts in LM Psychometrics](https://mlsanity.com/paper/2606.03357/) (2026)
- [From Representations to Behaviors: Exploring the Person-Situation-Behavior Triad in LLMs](https://mlsanity.com/paper/2607.26853/) (2026)
- [Which LLM Is Your Ideal Companion? Evaluating Emotional Companion Capabilities of LLMs Based on Adult Attachment Theory](https://mlsanity.com/paper/2608.13168/) (2026)
- [Faking Good and Faking Bad in LLMs: Response Distortion Across Dark Triad Personality Traits](https://mlsanity.com/paper/2609.17534/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly used in everyday life and research. One of the most common use cases is conversational interactions, enabled by the language generation capabilities of LLMs. Just as between two humans, a conversation between an LLM-powered entity and a human depends on the personality of the conversants. However, measuring the personality of a given LLM is currently a challenge. This paper introduces the Language Model Linguistic Personality Assessment (LMLPA), a system designed to evaluate the linguistic personalities of LLMs. Our system helps to understand LLMs' language generation capabilities by quantitatively assessing the distinct personality traits reflected in their linguistic outputs. Unlike traditional human-centric psychometrics, the LMLPA adapts a personality assessment questionnaire, specifically the Big Five Inventory, to align with the operational capabilities of LLMs, and also incorporates the findings from previous language-based personality measurement literature. To mitigate sensitivity to the order of options, our questionnaire is designed to be open-ended, resulting in textual answers. Thus, the AI rater is needed to transform ambiguous personality information from text responses into clear numerical indicators of personality traits. Utilising Principal Component Analysis and reliability validations, our findings demonstrate that LLMs possess distinct personality traits that can be effectively quantified by the LMLPA. This research contributes to Human-Computer Interaction and Human-Centered AI, providing a robust framework for future studies to refine AI personality assessments and expand their applications in multiple areas, including education and manufacturing.