mlsanityData to 14 Oct 2026

Machine individuality: Separating genuine idiosyncrasy from response bias in large language models

arXiv 2604.16755 ↗First posted 18 Apr 2026cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

As large language models (LLMs) are increasingly integrated into daily life, in roles ranging from high-stakes decision support to companionship, understanding their behavioral dispositions becomes critical. A growing literature uses psychometric inventories and cognitive paradigms to profile LLM dispositions. However, these approaches cannot determine whether behavioral differences reflect stable, stimulus-specific individuality or global response biases and stochastic noise. Here, we apply crossed random-effects models -- widely used in psychometrics to separate systematic effects -- to 74.9 million ratings provided by 10 open-weight LLMs for over 100,000 words across 14 psycholinguistic norms. On average, 16.9% of variance is attributable to stimulus-specific individuality, robustly exceeding a statistical null model. Cross-norm prediction analyses reveal this individuality as a coherent fingerprint, unique to each model. These results identify individual differences among LLMs that cannot be attributed to response biases or stochastic noise. We term these differences machine individuality.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 9 of its 30 references that are in the map. First posted on arXiv 18 Apr 2026.

Builds on

9 of 30 references.

References · 9

  1. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  2. 2022Using cognitive psychology to understand GPT-3cited by 398
  3. 2023Personality Traits in Large Language Modelscited by 169
  4. 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 141
  5. 2023Self-Assessment Tests are Unreliable Measures of LLM Personalitycited by 39
  6. 2025Idiosyncrasies in Large Language Modelscited by 33
  7. 2025Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests insteadcited by 7
  8. 2024Probing the contents of semantic representations from text, behavior, and brain data using the psychNorms metabasecited by 5
  9. 2026Failure of contextual invariance in large language modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.