Machine Psychometrics: A Mathematical Psychology of Artificial Intelligence
Not on the map. Why?
Abstract (arXiv)
Artificial agents now generate behavior rich enough to invite trust, surprise, and concern, yet our evaluation tools still privilege capability scores over psychological structure. This paper argues that the philosophical impasse between two symmetrical errors (Artificial Mind Blindness, which dismisses psychological organization in non-biological systems, and Artificial Mind Projection, which infers human-like inner life from fluent behavior alone) can be circumvented not by resolving the consciousness question, but by introducing a disciplined measurement layer beneath it. Drawing on Michael Levin's continuum view of cognition as goal-directed competency across substrates, and on the methodological repertoire of mathematical psychology (Item Response Theory, Signal Detection Theory, Bayesian cognitive modeling, calibration analysis, cognitive-bias batteries), the paper develops Machine Psychometrics as a measurement science of latent behavioral, metacognitive, communicative, and self-modeling dispositions in artificial agents. Its operational core is the Machine Mindprint: a multidimensional, domain-bounded, versioned profile spanning calibration, source integrity, suggestibility resistance, context stability, expressive alignment, tool integrity, drift monitoring, and distributional grounding. A complementary Trust Protocol turns Mindprints into deployment decisions through probe batteries, perturbation testing, reliability and validity analysis, and longitudinal monitoring across high-stakes domains. The philosophical contribution is a third stance, Artificial Mind Discipline, that neither anthropomorphizes nor dismisses, neither presupposes consciousness nor forecloses it. The aim is not to humanize artificial agents, but to understand them precisely because they are not human, through measurement before judgment.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 10 May 2026.
Builds on
19 of 28 references.
References · 19
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2023A Survey on Evaluation of Large Language Modelscited by 1,242
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
- 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
- 2023Do Large Language Models Know What They Don't Know?cited by 243
- 2023Consciousness in Artificial Intelligence: Insights from the Science of Consciousnesscited by 104
- 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 101
- 2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 98
- 2024A Framework for Human Evaluation of Large Language Models in Healthcare Derived from Literature Reviewcited by 66
- 2025Lost in Benchmarks? Rethinking Large Language Model Benchmarking with Item Response Theorycited by 45
- 2024Anchoring Bias in Large Language Models: An Experimental Studycited by 37
- 2025TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Modelscited by 33
7 more
- 2025Language Models Fail to Introspect About Their Knowledge of Languagecited by 30
- 2021Technological Approach to Mind Everywhere (TAME): an experimentally-grounded framework for understanding diverse bodies and mindscited by 25
- 2025Metacognition and Uncertainty Communication in Humans and Large Language Modelscited by 23
- 2024Evaluating Large Language Models with Psychometricscited by 22
- 2021Item Response Theory -- A Statistical Framework for Educational and Psychological Measurementcited by 17
- 2025Latency-Response Theory Model: Evaluating Large Language Models via Response Accuracy and Chain-of-Thought Lengthcited by 5
- 2026The Surprising Universality of LLM Outputs: A Real-Time Verification Primitivecited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026RAILS: Verification-Native Clearing For Agentic Commercecited by 3
- 2026RAILS: Verification-Native Clearing For Agentic Commerce7 Jun 2026