# Mind Reading or Misreading? LLMs on the Big Five Personality Test

arXiv: https://arxiv.org/abs/2511.23101 · HTML: https://mlsanity.com/paper/2511.23101/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [Generative AI](https://mlsanity.com/paper/2309.07930/) (2023)
- [PANDORA Talks: Personality and Demographics on Reddit](https://mlsanity.com/paper/2004.04460/) (2020)
- [Systematic Evaluation of GPT-3 for Zero-Shot Personality Estimation](https://mlsanity.com/paper/2306.01183/) (2023)
- [Is ChatGPT a Good Personality Recognizer? A Preliminary Study](https://mlsanity.com/paper/2307.03952/) (2023)
- [Text-Based Automatic Personality Prediction Using KGrAt-Net; A Knowledge Graph Attention Network Classifier](https://mlsanity.com/paper/2205.13780/) (2022)
- [Large Language Models Can Infer Personality from Free-Form User Interactions](https://mlsanity.com/paper/2405.13052/) (2024)
- [Predicting the Big Five Personality Traits in Chinese Counselling Dialogues Using Large Language Models](https://mlsanity.com/paper/2406.17287/) (2024)
- [Can ChatGPT Read Who You Are?](https://mlsanity.com/paper/2312.16070/) (2023)
- [Investigating Large Language Models in Inferring Personality Traits from User Conversations](https://mlsanity.com/paper/2501.07532/) (2025)

## Abstract (arXiv)

We evaluate large language models (LLMs) for automatic personality prediction from text under the binary Five Factor Model (BIG5). Five models -- including GPT-4 and lightweight open-source alternatives -- are tested across three heterogeneous datasets (Essays, MyPersonality, Pandora) and two prompting strategies (minimal vs. enriched with linguistic and psychological cues). Enriched prompts reduce invalid outputs and improve class balance, but also introduce a systematic bias toward predicting trait presence. Performance varies substantially: Openness and Agreeableness are relatively easier to detect, while Extraversion and Neuroticism remain challenging. Although open-source models sometimes approach GPT-4 and prior benchmarks, no configuration yields consistently reliable predictions in zero-shot binary settings. Moreover, aggregate metrics such as accuracy and macro-F1 mask significant asymmetries, with per-class recall offering clearer diagnostic value. These findings show that current out-of-the-box LLMs are not yet suitable for APPT, and that careful coordination of prompt design, trait framing, and evaluation metrics is essential for interpretable results.