# Simulated Self-Assessment in Large Language Models: A Psychometric Approach to AI Self-Efficacy

arXiv: https://arxiv.org/abs/2511.19872 · HTML: https://mlsanity.com/paper/2511.19872/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 79 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Builds on (most cited first, 24 of 24)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark](https://mlsanity.com/paper/2310.18018/) (2023)
- [LiveBench: A Challenging, Contamination-Limited LLM Benchmark](https://mlsanity.com/paper/2406.19314/) (2024)
- [LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts](https://mlsanity.com/paper/2407.04973/) (2024)
- [ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning](https://mlsanity.com/paper/2502.01100/) (2025)
- [Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs](https://mlsanity.com/paper/2311.04892/) (2023)
- [ELEPHANT: Measuring and understanding social sycophancy in LLMs](https://mlsanity.com/paper/2505.13995/) (2025)
- [Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad](https://mlsanity.com/paper/2503.21934/) (2025)
- [Internal Consistency and Self-Feedback in Large Language Models: A Survey](https://mlsanity.com/paper/2407.14507/) (2024)
- [Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement](https://mlsanity.com/paper/2505.08245/) (2025)
- [Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs](https://mlsanity.com/paper/2305.14693/) (2023)
- [History, Development, and Principles of Large Language Models-An Introductory Survey](https://mlsanity.com/paper/2402.06853/) (2024)
- [Strong and weak alignment of large language models with human values](https://mlsanity.com/paper/2408.04655/) (2024)
- [RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios](https://mlsanity.com/paper/2412.08972/) (2024)
- [EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code](https://mlsanity.com/paper/2505.13004/) (2025)
- [FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs](https://mlsanity.com/paper/2410.13210/) (2024)
- [Evaluating LLM Metrics Through Real-World Capabilities](https://mlsanity.com/paper/2505.08253/) (2025)
- [The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models](https://mlsanity.com/paper/2404.05904/) (2024)
- [PHAnToM: Persona-based Prompting Has An Effect on Theory-of-Mind Reasoning in Large Language Models](https://mlsanity.com/paper/2403.02246/) (2024)
- [Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing](https://mlsanity.com/paper/2507.10587/) (2025)
- [Pitfalls of Evaluating Language Models with Open Benchmarks](https://mlsanity.com/paper/2507.00460/) (2025)

## Built on by (most cited first, 1 of 1)

- [Quantitative Introspection in Language Models: Tracking Emotive States Across Conversation](https://mlsanity.com/paper/2603.18893/) (2026)

## Abstract (arXiv)

Large language model (LLM) proficiency in evaluating and quantifying their capacities remains uncertain. We conducted a controlled psychometric measurement study adapting the 10-item General Self-Efficacy Scale (GSES) to evaluate simulated self-assessment across 10 contemporary LLMs. Models completed the GSES under a no-task control condition and after three task-priming conditions: computational reasoning, social reasoning, and summarization. We examined between-model variation, task versus no-task differences, within-model stability, item-order robustness, internal consistency, and qualitative reasoning patterns. LLM-generated GSES responses were highly stable across repeated administrations, with nearly all model-task-item scores remaining identical across three runs. Internal consistency was high across task conditions, with Cronbachs alpha , and item-order effects were minimal, with intraclass correlation coefficients . Despite this stability, simulated self-efficacy differed significantly between models across all task and no-task conditions . Composite LLM GSES scores were lower and more variable than human norms (compared to the literature), and the latent response structure did not replicate human self-efficacy patterns. Qualitative analysis suggested that models differed in how they interpreted GSES items involving effort, coping, agency, and persistence. Some models rejected these constructs as inapplicable to artificial systems, while others translated them into task-oriented capability language. These findings indicate that LLMs can generate internally consistent psychometric self-assessments, but these outputs appear to reflect model-specific and context-sensitive communication behavior rather than human-like self-efficacy or metacognition. Psychometric prompting may help characterize how LLMs express capability, limitation, and agency under different prompting contexts.