# Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications

arXiv: https://arxiv.org/abs/2505.00049 · HTML: https://mlsanity.com/paper/2505.00049/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 47 of its 136 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 30 of 47)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://mlsanity.com/paper/2110.07178/) (2021)
- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [Do LLMs exhibit human-like response biases? A case study in survey design](https://mlsanity.com/paper/2311.04076/) (2023)
- [User Behavior Simulation with Large Language Model based Agents](https://mlsanity.com/paper/2306.02552/) (2023)
- [SmartPlay: A Benchmark for LLMs as Intelligent Agents](https://mlsanity.com/paper/2310.01557/) (2023)
- [GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations](https://mlsanity.com/paper/2402.12348/) (2024)
- [Who is GPT-3? An Exploration of Personality, Values and Demographics](https://mlsanity.com/paper/2209.14338/) (2022)
- [GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents](https://mlsanity.com/paper/2406.06613/) (2024)
- [Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis](https://mlsanity.com/paper/2312.05488/) (2023)
- [MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming](https://mlsanity.com/paper/2310.06500/) (2023)
- [Do LLM Agents Exhibit Social Behavior?](https://mlsanity.com/paper/2312.15198/) (2023)
- [BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data](https://mlsanity.com/paper/2410.16491/) (2024)
- [Estimating the Personality of White-Box Language Models](https://mlsanity.com/paper/2204.12000/) (2022)
- [Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Tests](https://mlsanity.com/paper/2310.20320/) (2023)
- [Evaluating Language Model Agency through Negotiations](https://mlsanity.com/paper/2401.04536/) (2024)
- [Identifying and Manipulating the Personality Traits of Language Models](https://mlsanity.com/paper/2212.10276/) (2022)
- [Self-Assessment Tests are Unreliable Measures of LLM Personality](https://mlsanity.com/paper/2309.08163/) (2023)
- [Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networks](https://mlsanity.com/paper/2406.17232/) (2024)
- [Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs](https://mlsanity.com/paper/2305.14693/) (2023)
- [Limited Ability of LLMs to Simulate Human Psychological Behaviours: a Psychometric Analysis](https://mlsanity.com/paper/2405.07248/) (2024)
- [Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench](https://mlsanity.com/paper/2310.01386/) (2023)
- [LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output](https://mlsanity.com/paper/2411.06032/) (2024)
- [Evaluating Personality Traits in Large Language Models: Insights from Psychological Questionnaires](https://mlsanity.com/paper/2502.05248/) (2025)
- [LLMs Simulate Big Five Personality Traits: Further Evidence](https://mlsanity.com/paper/2402.01765/) (2024)
- [LLMs with Personalities in Multi-issue Negotiation Games](https://mlsanity.com/paper/2405.05248/) (2024)
- [Do LLMs Exhibit Human-Like Reasoning? Evaluating Theory of Mind in LLMs for Open-Ended Responses](https://mlsanity.com/paper/2406.05659/) (2024)

## Built on by (most cited first, 10 of 10)

- [PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra](https://mlsanity.com/paper/2602.15669/) (2026)
- [Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigations](https://mlsanity.com/paper/2505.15392/) (2025)
- [From Five Dimensions to Many: Large Language Models as Precise and Interpretable Psychological Profilers](https://mlsanity.com/paper/2511.03235/) (2025)
- [PERM: Psychology-grounded Empathetic Reward Modeling for Large Language Models](https://mlsanity.com/paper/2601.10532/) (2026)
- [ARISE: Agentic Rubric-Guided Iterative Survey Engine for Automated Scholarly Paper Generation](https://mlsanity.com/paper/2511.17689/) (2025)
- [AI-exhibited Personality Traits Can Shape Human Self-concept through Conversations](https://mlsanity.com/paper/2601.12727/) (2026)
- [ZPD-SCA: Unveiling the Blind Spots of LLMs in Assessing Students' Cognitive Abilities](https://mlsanity.com/paper/2508.14377/) (2025)
- [Value-Action Alignment in Large Language Models under Privacy-Prosocial Conflict](https://mlsanity.com/paper/2601.03546/) (2026)
- [From Empathy to Personalized Empathy: Adapting Empathetic Strategies to Individual Users](https://mlsanity.com/paper/2606.00728/) (2026)
- [When Personality Meets Quantization: A Layer-wise MBTI Analysis of Quantized LLMs](https://mlsanity.com/paper/2608.25977/) (2026)

## Abstract (arXiv)

As large language models (LLMs) are increasingly used in human-centered tasks, assessing their psychological traits is crucial for understanding their social impact and ensuring trustworthy AI alignment. While existing reviews have covered some aspects of related research, several important areas have not been systematically discussed, including detailed discussions of diverse psychological tests, LLM-specific psychological datasets, and the applications of LLMs with psychological traits. To address this gap, we systematically review six key dimensions of applying psychological theories to LLMs: (1) assessment tools; (2) LLM-specific datasets; (3) evaluation metrics (consistency and stability); (4) empirical findings; (5) personality simulation methods; and (6) LLM-based behavior simulation. Our analysis highlights both the strengths and limitations of current methods. While some LLMs exhibit reproducible personality patterns under specific prompting schemes, significant variability remains across tasks and settings. Recognizing methodological challenges such as mismatches between psychological tools and LLMs' capabilities, as well as inconsistencies in evaluation practices, this study aims to propose future directions for developing more interpretable, robust, and generalizable psychological assessment frameworks for LLMs.