# The Shared Latent Structure of LLM Psychometric Responses

arXiv: https://arxiv.org/abs/2605.05080 · HTML: https://mlsanity.com/paper/2605.05080/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 37 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 21 of 21)

- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [Looking Inward: Language Models Can Learn About Themselves by Introspection](https://mlsanity.com/paper/2410.13787/) (2024)
- [Who is GPT-3? An Exploration of Personality, Values and Demographics](https://mlsanity.com/paper/2209.14338/) (2022)
- [Tell me about yourself: LLMs are aware of their learned behaviors](https://mlsanity.com/paper/2501.11120/) (2025)
- [The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models](https://mlsanity.com/paper/2601.10387/) (2026)
- [Moral Foundations of Large Language Models](https://mlsanity.com/paper/2310.15337/) (2023)
- [Could a Large Language Model be Conscious?](https://mlsanity.com/paper/2303.07103/) (2023)
- [Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics](https://mlsanity.com/paper/2406.14703/) (2024)
- [Large Language Models Show Human-like Social Desirability Biases in Survey Responses](https://mlsanity.com/paper/2405.06058/) (2024)
- [Challenging the Validity of Personality Tests for Large Language Models](https://mlsanity.com/paper/2311.05297/) (2023)
- [Does It Make Sense to Speak of Introspection in Large Language Models?](https://mlsanity.com/paper/2506.05068/) (2025)
- [Value Portrait: Assessing Language Models' Values through Psychometrically and Ecologically Valid Items](https://mlsanity.com/paper/2505.01015/) (2025)
- [Personality testing of Large Language Models: Limited temporal stability, but highlighted prosociality](https://mlsanity.com/paper/2306.04308/) (2023)
- [Self-assessment, Exhibition, and Recognition: a Review of Personality in Large Language Models](https://mlsanity.com/paper/2406.17624/) (2024)
- [From Prompts to Constructs: A Dual-Validity Framework for Large Language Model Research in Psychology](https://mlsanity.com/paper/2506.16697/) (2025)
- [Political Alignment in Large Language Models: A Multidimensional Audit of Psychometric Identity and Behavioral Bias](https://mlsanity.com/paper/2601.06194/) (2026)
- [A Detailed Factor Analysis for the Political Compass Test: Navigating Ideologies of Large Language Models](https://mlsanity.com/paper/2506.22493/) (2025)
- [On the Credibility of Evaluating LLMs using Survey Questions](https://mlsanity.com/paper/2602.04033/) (2026)
- [Interpretable Semantic Gradients in SSD: A PCA Sweep Approach and a Case Study on AI Discourse](https://mlsanity.com/paper/2603.13038/) (2026)

## Abstract (arXiv)

Human psychometric questionnaires are increasingly used to characterize large language models (LLMs), but it remains unclear what exactly their responses reflect. To answer this question we analyze responses from 50 LLMs across 45 questionnaires spanning multiple psychological domains using psychometric methodology. We found a dominant principal component explaining 47.1% of variance across standardized, response-derived questionnaire-level primary scores, revealing substantial shared structure across instruments. We investigate its meaning through semantic analysis, respondent framing, and human annotations. Semantic analysis suggests that the component partly reflects experiential self-attribution: how readily models endorse statements describing subjective states and bodily experiences. A model score emphasizing items whose disagreement decreases under human simulation correlates strongly with the component (r=0.864). Human ratings of 289 matched items show that inner-life content is positively associated, and social desirability negatively associated, with item-component correlations after mutual adjustment and controls for other semantic properties. Together, six rated properties explain 22.1% of variance in Fisher-transformed item-component correlations, supporting a partial interpretation involving both experiential self-attribution and socially desirable responding. These findings suggest that LLM psychometric profiles share response tendencies that require interpretation beyond the human traits their questionnaires were designed to measure.