Evaluating Developmental Cognition Capabilities of LLMs
Not on the map. Why?
Abstract (arXiv)
Conversational AI is increasingly personalized around users' preferences, histories, goals, and knowledge, but much less around how users interpret and take up model outputs to construct and understand their reality. We draw on Robert Kegan's constructive-developmental theory as a complementary lens on this dimension. Existing methods for assessing developmental stage in the Keganian tradition rely either on expert interviews that do not scale or on sentence-completion instruments that are proprietary, lengthy, or invasive. To make this perspective tractable for LLM evaluation, we introduce the Developmental Sentence Completion Test (DSCT), a 20-item instrument designed to elicit developmental signal in self-administered text. Throughout, we treat the resulting labels as characterizations of stage-like structure in elicited responses, not as validated person-level developmental stage. We then ask how much of that signal can be recovered by LLMs across three elicited response regimes: simulated personas, real human respondents, and default model-generated answers. On simulated personas, top frontier models recover simulator-intended labels with high accuracy. On real human DSCT responses, human-LLM agreement is fair, with much stronger within-neighborhood than exact agreement. Finally, when LLMs answer DSCT prompts without persona-conditioning, their responses exhibit stable stage-like differences across model families, with larger and newer models tending to generate higher-rated text. These results suggest that stage-conditioned signal is cleaner in synthetic responses than in human-written DSCT text, and that the core constraint for stage-aware conversational AI is not classifier accuracy alone, but the availability of developmental signal from elicited text.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 7 of its 33 references that are in the map. First posted on arXiv 8 May 2026.
Builds on
7 of 33 references.
References · 7
- 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
- 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
- 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 89
- 2024A Computational Framework for Behavioral Assessment of LLM Therapistscited by 61
- 2026Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesianscited by 20
- 2025Do Repetitions Matter? Strengthening Reliability in LLM Evaluationscited by 14
- 2025Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistancecited by 6
Built on by
Not cited yet.