# GenPT: Beyond Self-Report for Reliable LLM Psychometrics via Generative Projective Testing

arXiv: https://arxiv.org/abs/2606.00860 · HTML: https://mlsanity.com/paper/2606.00860/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 30 May 2026.

## Builds on (most cited first, 11 of 11)

- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [From Persona to Personalization: A Survey on Role-Playing Language Agents](https://mlsanity.com/paper/2404.18231/) (2024)
- [InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews](https://mlsanity.com/paper/2310.17976/) (2023)
- [Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement](https://mlsanity.com/paper/2505.08245/) (2025)
- [D4: a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chat](https://mlsanity.com/paper/2205.11764/) (2022)
- [Evaluating Personality Traits in Large Language Models: Insights from Psychological Questionnaires](https://mlsanity.com/paper/2502.05248/) (2025)
- [Customizing Emotional Support: How Do Individuals Construct and Interact With LLM-Powered Chatbots](https://mlsanity.com/paper/2504.12943/) (2025)
- [The influence of persona and conversational task on social interactions with a LLM-controlled embodied conversational agent](https://mlsanity.com/paper/2411.05653/) (2024)
- [A Systematic Analysis of the Impact of Persona Steering on LLM Capabilities](https://mlsanity.com/paper/2604.11048/) (2026)
- [Dynamic Context Adaptation for Consistent Role-Playing Agents with Retrieval-Augmented Generations](https://mlsanity.com/paper/2508.02016/) (2025)

## Built on by (most cited first, 3 of 3)

- [Do AI Personas Grow? Analyzing and Benchmarking Personality Evolution in LLM Agents After Life Events](https://mlsanity.com/paper/2608.06485/) (2026)
- [LongCounsel-8: A Benchmark Suite for Longitudinal Depression Tracking from Multi-Session Counseling Dialogues](https://mlsanity.com/paper/2609.03507/) (2026)
- [From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Health](https://mlsanity.com/paper/2609.25186/) (2026)

## Abstract (arXiv)

Self-report questionnaires remain the prevailing tool for probing the psychological states of persona-conditioned agents (PC-Agents). However, classical instruments inherit two well-known threats: contamination from training corpora and directional bias driven by social-desirability or contextual framing. To overcome these methodological bottlenecks, we ask whether projective paradigms can be adapted into a robust psychometric tool. We introduce GenPT (Generative Projective Testing), which reformulates TAT, Rorschach, and SCT with newly generated stimuli and organizes assessment as a three-stage pipeline to derive standardized psychological indicators and target states. Evaluating PC-Agents induced via CharacterRAG and AnnaAgent profiles, we benchmark GenPT's reliability and validity against classical questionnaires. The results indicate that questionnaires exhibit systematic directional shifts under social-desirability framing, most strongly on suicide ideation. In contrast, GenPT's collected behavioral patterns stay near the symmetric baseline. Furthermore, under a longitudinal counselling context, GenPT-based depression assessment shifts by roughly an order of magnitude more than the questionnaire counterpart when Qwen3 serves as the backbone. Overall, GenPT complements self-report methods in scenarios where contamination resistance, bias asymmetry, and context sensitivity matter. Code and stimuli can be found at https://github.com/sci-m-wang/GenPT.