# Assessment and manipulation of latent constructs in pre-trained language models using psychometric scales

arXiv: https://arxiv.org/abs/2409.19655 · HTML: https://mlsanity.com/paper/2409.19655/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 55 references that are in the map.
- First posted on arXiv 29 Sep 2024.

## Builds on (most cited first, 10 of 10)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Personality Traits in Large Language Models](https://mlsanity.com/paper/2307.00184/) (2023)
- [Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models](https://mlsanity.com/paper/2307.16180/) (2023)
- [Estimating the Personality of White-Box Language Models](https://mlsanity.com/paper/2204.12000/) (2022)
- [Identifying and Manipulating the Personality Traits of Language Models](https://mlsanity.com/paper/2212.10276/) (2022)
- [Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs](https://mlsanity.com/paper/2305.14693/) (2023)
- [Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench](https://mlsanity.com/paper/2310.01386/) (2023)
- [Four Years in Review: Statistical Practices of Likert Scales in Human-Robot Interaction Studies](https://mlsanity.com/paper/2001.03231/) (2020)

## Built on by (most cited first, 7 of 7)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [A Review of Incorporating Psychological Theories in LLMs](https://mlsanity.com/paper/2505.00003/) (2025)
- [Responsible Evaluation of AI for Mental Health](https://mlsanity.com/paper/2602.00065/) (2026)
- [FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment](https://mlsanity.com/paper/2604.04992/) (2026)
- [MindShift: Analyzing Language Models' Reactions to Psychological Prompts](https://mlsanity.com/paper/2512.09149/) (2025)
- [Projective Psychological Assessment of Large Multimodal Models Using Thematic Apperception Tests](https://mlsanity.com/paper/2602.17108/) (2026)
- [Trust Stack for Mental Health AI: A Survey of Calibration across Human, Interaction, and AI Layers](https://mlsanity.com/paper/2604.20166/) (2026)

## Abstract (arXiv)

Human-like personality traits have recently been discovered in large language models, raising the hypothesis that their (known and as yet undiscovered) biases conform with human latent psychological constructs. While large conversational models may be tricked into answering psychometric questionnaires, the latent psychological constructs of thousands of simpler transformers, trained for other tasks, cannot be assessed because appropriate psychometric methods are currently lacking. Here, we show how standard psychological questionnaires can be reformulated into natural language inference prompts, and we provide a code library to support the psychometric assessment of arbitrary models. We demonstrate, using a sample of 88 publicly available models, the existence of human-like mental health-related constructs (including anxiety, depression, and Sense of Coherence) which conform with standard theories in human psychology and show similar correlations and mitigation strategies. The ability to interpret and rectify the performance of language models by using psychological tools can boost the development of more explainable, controllable, and trustworthy models.